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ABSTRACT 

This study examines how the federal government and states 
prepare to implement the No Child Left Behind Act. Researchers reviewed 
states’ consolidated applications for federal grants; conducted telephone 
interviews with officials in 48 states, the District of Columbia, and 8 
school districts; and reviewed relevant research. The study addresses 
assessment and accountability; public school choice and supplemental 
education services requirements; teacher and paraprofessional quality; and 
use of scientifically based research to improve education. Overall, states 
are committed to the Act’s goals of raising achievement for all subgroups and 
enhancing teacher quality and working to implement the law, though its 
deadlines for making massive changes are short and final regulations on some 
key issues were late in coming from the Department of Education. The main 
challenge states face are creating plans for determining whether schools are 
making adequate yearly progress in raising achievement. States and districts 
are slow in acting on requirements to offer public school choice and 
supplemental services. They are not very far along in using scientifically 
based research to inform classroom practice. Three appendices include 
protocol for review of state plans, an interview guide, and a policy summary 
of the No Child Left Behind Act. (Chapters contain references.) (SM) 
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Commentary 



A year ago, in January 2002, President Bush signed the No Child Left Behind 
Act. This far-reaching education law has two major purposes: to raise student 
achievement across the board and to eliminate the achievement gap between 
students from different backgrounds. To accomplish these ends, the law requires states 
and school districts to deal forcefully with schools that are not improving achievement 
and to ensure that all students are taught by highly qualified teachers. In our opinion, 
these are worthy goals for the country. 

Over a five-month period during the summer and fall of 2002, the Center on 
Education Policy reviewed die implementation of this Act by the U.S. Department of 
Education and the state governments. We read the plans states submitted to the federal 
government, conducted confidential interviews with scores of state administrators in 
nearly every state, and reviewed the guidance and regulations issued by the federal gov- 
ernment. During this first year of what is intended to be a six-year study, we concen- 
trated on the federal and state actors, because the main tasks required by the Act for 
2002 involved building state-level systems for additional testing, greater accountability, 
and data collection, with only a few new requirements affecting some local schools. In 
the fall of 2003, however, this will change, as the force of this major challenge for 
American public education reaches every local school in the country. 

Our study found that the states are committed to the goals of the legislation and 
are trying hard to carry them out, but the prescriptive nature of the requirements is 
causing great concern. States are moving faster on the elements of the law where they 
have more experience, such as developing state tests, and slower on aspects where they 
need to create new procedures, such as approving nonprofit and for-profit groups to 
provide tutoring. We also found that the fiscal crisis in most states, coupled with the 
prospect of limited additional federal aid, could threaten the successful implementation 
of this very ambitious law. ° 0 ° 

Nearly four decades ago, another ambitious federal policy was initiated when v 

President Johnson signed into law the Elementary and Secondary Education Act 
(ESEA) of 1965. This legislation, which vastly expanded federal aid to education, was 
meant to improve American public schools and help poor children climb out of pover- 
ty by providing them with a better education. New compensatory education programs 
took shape in almost every school district, new textbooks were bought, innovative pro- 
grams were offered, and research was conducted to find better ways of teaching and 
learning. There was great enthusiasm for that law because its promises seemed so noble. 

Four decades after ESEA was enacted, debate persists about the original Act s effec- 
tiveness in improving American schools and raising the achievement of poor children, 
although the weight of evidence suggests its impact on student learning was modest. In 
short, a major policy was not as successful as it could have been, because the promises 
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made by President Johnson and his supporters were overstated and the changes need- 
ed to carry out those promises were underestimated. Moreover, the war in Vietnam 
drew funding and attention away from the Act, slowing its momentum. In the years that 
followed, many poor children received additional services as a result of ESEA, but the 
ways of administering its programs became too highly regulated and perfunctory 

After completing our analysis of the first year of the implementation of the No 
Child Left Behind Act (NCLB), we see striking — and unsettling — parallels with the 
first years of the Elementary and Secondary Education Act, even though the times are 
obviously different. President Bush and Members of Congress have made lofty prom- 
ises for the success of this initiative, while generally underestimating the magnitude of 
change that must occur in American public education to bring about those promises. 
Although President Bush supported a significant increase in appropriations for the first 
year of NCLB as part of a political compromise to get the measure enacted, his budg- 
et for the second year proposes a much smaller increase. The states are in no financial 
shape to make up the funding difference. And since the Act was signed, the nation’s 
attention and resources have shifted to a war on terrorism and now a possible war 
against Iraq. 

Some critics of the original ESEA say that it failed because it provided money 
without accountability, and that the NCLB Act will succeed because it requires strict 
accountability. The ESEA of 1965 may have offered money without much educational 
accountability, but the NCLB Act demands heavy accountability without much greater 
federal financial and technical assistance — an approach no more likely to succeed. 

In 1965, extensive federal requirements like those in NCLB would never have made 
it through the Congress. At that time, the federal role in education was marginal, most 
state education agencies had very limited authority and capabilities, and local people 
were extremely wary that more federal aid would bring federal control. Since then, the 
federal and state roles in education have grown, and states and school districts recognize 
that accepting federal requirements goes along with receiving federal funding. 

The No Child Left Behind Act places greater demands on states and school 
districts than ever before. States must define the level of proficiency that all students are 
expected to reach and set a timetable for schools to bring all their students up to this 
level by school year 2013-14. States must also expand their testing programs, analyze 
and report test results in new ways, provide technical assistance to under-performing 
districts and schools, help teachers become better qualified, and much more. School dis- 
tricts must raise test scores in reading, math, and science, close achievement gaps, design 
improvement strategies and interventions for under-performing schools, hire or devel- 
op better-qualified teachers and classroom aides, and create or expand public school 
choice programs, among other duties. 

It remains to be seen how well these demands will be accepted and carried out. 
Since the federal government continues to provide only about 7% of the total funding 
for public elementary and secondary schools, it may have trouble demanding 100% 
accountability from schools. 
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We want this law to succeed because its goals are the right ones for American chil- 
dren. By spotlighting the performance of poor students, disabled students, and students 
from all major racial-ethnic groups, the law refines the commitment to disadvantaged 
children that began in 1965, while also recognizing the need to provide a high-quality 
education for all children. These goals resonate with people, as we heard in our inter- 
views during the last months. We therefore offer these recommendations for this sec- 
ond year of the Act. 

First, the rhetoric from Washington ought to be toned down. The President and 
others ought to be careful about making grand promises for NCLB, because the chal- 
lenges of carrying them out will soon become very clear. Our leaders should empha- 
size the soundness of these goals but also recognize how much work it will take at the 
state and local levels to accomplish them. 

Second, the federal government ought to be flexible and sensitive to the complex 
and varying circumstances found in states and school districts — which, after all, provide 
the vast majority of education funding. To the extent possible, the Department should 
allow states to achieve the goals of the law without having to dismantle aspects of their 
assessment and accountability systems that are working well. If the U.S. Department of 
Education takes a rigid, heavy-handed approach to compliance, our state interviews 
suggest that state and local responses could turn from dedication to making the law 
work to mere technical compliance with the law’s many detailed requirements. 

Third, the President and the Congress ought to fully fund the Act so states and 
school districts can go about making improvements. The U. S. Department of Education 
should also provide technical assistance to help states and districts find ways to raise stu- 
dent achievement through such means as improved professional development for teach- 
ers and more effective strategies to help students. 

Fourth, all the requirements of the law ought to be implemented as fully as possi- 
ble. National policymakers should carefully monitor how the requirements play out in 
states, schools, and classrooms. If unintended effects emerge, then changes in the law 
should be considered in later years. For example, if large numbers of schools are tar- 
geted for interventions because of test score volatility unrelated to student learning, as 
explained in Chapter 2 of our report, then policymakers should consider changes that 
would target interventions and assistance on schools with the highest degree of failure 
and greatest needs. 

The No Child Left Behind Act is too important to America’s children for the 
nation to let it fail. We urge our leaders to closely monitor the impact of the legisla- 
tion, be ready to make changes if needed, and commit themselves to providing the 
major financial and other assistance that will be essential to help every child to succeed. 



Jack Jennings 

Director, Center on Education Policy 
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Chapter 1: Looking Broadly at the Act's First Year 



After reviewing the first year of implementation of the No Child Left Behind Act, the 
Center on Education Policy has concluded that states are committed to the Act s goals 
of raising achievement for all subgroups of students and enhancing the quality of the 
teaching force. States are trying hard to implement the law, even though its deadlines 
for making massive changes are short and the final regulations on a few key issues were 
late in coming from the U.S. Department of Education (USED). The Center also 
reached other broad conclusions about federal and state implementation: 

m The law contains many detailed requirements, and support for it could erode if 
the U.S. Department of Education applies these requirements too rigidly — for 
example, by preventing states from maintaining elements of their existing testing 
and accountability systems that are working well. Limited waivers of specific 
requirements may be necessary in some individual cases to accomplish the law’s 
broader goals. 

States are making headway in carrying out the law’s requirements for standards and 
assessment, because they already have experience in these areas, but are moving 
more slowly on implementing newer roles, such as developing fists of approved 
providers of supplemental education services, helping local school districts assess 
the qualifications of Title I instructional aides, or providing guidance to school dis- 
tricts about which instructional practices are based on scientific research. 

H The majority of states interviewed by the Center rated the quality of USED writ- 
ten regulations and guidance as good to fair, with guidance on assessments and 
teacher quality receiving higher ratings than guidance on scientifically based 
research. States gave the Department lower marks for the timeliness of its regula- °o° 

tions and guidance. . 

■ The Act is asking states to do more than ever before at a time when most states are 
facing severe financial strains. If federal education funding for fiscal year 2003 pro- 
vides only the modest increase proposed in the President’s budget, states and school 
districts will be hard pressed to carry out the many new demands, such as provid- 
ing technical assistance to all schools identified for improvement. 
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Chapter 2: Assessing Student Progress 
and Holding Schools Accountable 
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The assessment and accountability provisions of NCLB.are the heart of the Act. The 

Centers study revealed the following about these aspects of implementation: 

□ The most formidable challenge of the law, according to virtually all state officials 
interviewed, is to come up with a suitable plan for determining whether schools 
are making adequate yearly progress in raising student achievement. State efforts to 
do this have been hampered by the federal government’s delay in issuing final reg- 
ulations on adequate yearly progress and other complex accountability questions. 

D The rules for determining whether schools and districts are making adequate year- 
ly progress present practical and technical dilemmas that could result in large num- 
bers of schools being identified for improvement. States are concerned that if they 
have to assist all of the schools identified, it will spread their resources too thinly 
and subvert the laws intent of targeting extra money and help on the schools and 
students that need it most. 

0 NCLB continues to allow each state to define proficiency in its own way, which 
means that states that set higher expectations could be unintentionally penalized, 
because their schools will have farther to go to reach the goal of 100% proficien- 
cy, while states with lower expectations could have an incentive to keep them low. 



Chapter 3: Providing Public School Choice 
and Supplemental Education Services 

Among the provisions of NCLB that affected school districts this fall were require- 
ments to offer public school choice and supplemental education services, such as tutor- 
ing, to students in schools identified as needing improvement under the terms of the 
prior law. States and school districts were generally slow in acting on these require- 
ments, according to our state interviews and other anecdotal evidence. Many school 
districts had not yet received test data showing which schools and students would be 
eligible, and states were awaiting final regulations from the U.S. Department of Educa- 
tion. Other findings also emerged from the Centers study: 

□ In fall 2002, several states estimated that few parents had made use of the public 
school choice option thus far. This could be due to state and local delays in imple- 
menting the requirements, inadequate efforts to inform parents that the option was 
available, or the commitment of parents to their neighborhood schools. 

□ Even with the declaration in final regulations that lack of capacity cannot be used 
to deny choice for students, practical constraints could make it difficult to imple- 
ment the choice requirements in districts that have overcrowded schools, have a 
single school at certain grade spans, or are located in rural or remote areas. 




■ As of November 2002, only 15 states had developed a list of approved providers of 
supplemental education services, as required by Department of Education guid- 
ance. Many states were waiting for final regulations, while others did not have any 
schools that were required to offer these services this school year. The number of 
providers on state-approved lists varied widely, from a handful to more than 100. 



Chapter 4: Improving the Qualifications 
of Teachers and Paraprofessionals 



The NCLB Act requires all teachers of core academic subjects to be “highly qualified,” 
as defined in the law, and raises the minimum qualifications for paraprofessionals in Ti- 
tle I schools. The Center reached several findings about implementation of these provi- 
sions, which have already taken effect for some teachers and paraprofessionals: 

■ All the states we interviewed had or would soon have information on the number 
of teachers who are fully certified, but many said their data gathering systems will 
have to be revised to fully reflect the federal definition of“highly qualified.” States 
were less far along in collecting data about the qualifications of paraprofessionals; 
most states will need to start from scratch to develop data systems about parapro- 

. fessionals or will rely on school districts to do it. 

■ Whether the NCLB teacher and paraprofessional qualification requirements will 
help to improve instruction depends on both the rigor of states’ teacher certifica- 
tion requirements and the evaluation instruments used to assess the knowledge and 
skills of current teachers and paraprofessionals. 



Chapter 5: Using Scientifically Based 
Research to Improve Education °o 0 o° 

xi 

Numerous programs in the No Child Left Behind Act require schools to use scientifi- 
cally based research to inform their classroom practices. States are not as far along in 
implementing these requirements as they are with other major aspects of the law. This 
seems to be mostly because several states were waiting on the federal government to 
provide more definitive guidance, and a few felt they lacked expertise on this issue. 

State officials interviewed gave the Department low marks for its overall guidance, or 
lack thereof, on scientifically based research. The Center’s review also reached these 
findings about the provisions for scientifically based research: 
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□ Many Tide I state directors report that the federally funded Reading First program 
in their states is taking the lead in carrying out requirements to use scientifically 
based research. Since the Reading First program uses a special definition of scien- 
tifically based research that is less rigorous than the general definition that applies 
to other NCLB programs, it may not be prudent for Title I programs to rely on 
Reading First materials. 

d Some state officials, researchers, and educators are concerned that strict adherence 
to the law s general definition of scientifically based research will produce few pro- 
grams or approaches that meet it or could severely limit classroom teaching meth- 
ods and materials. 
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CHAPTER 1 



Looking Broadly 
at the Act’s First Year 

KEY FINDINGS 

B States are generally committed to the goals of the No Child Left Behind Act 
(NCLB) and are trying hard to implement it, even though the law's deadlines for 
making massive changes are short and the final regulations on such key issues as 
"adequate yearly progress" were late in coming from the U.S. Department of 
Education (USED). 

B A stated principle of the Act is to allow greater flexibility in federal education 
programs in exchange for increased accountability. But this philosophy could be 
undercut by the law's many highly specific requirements, the strict interpreta- 
tions in the final regulations, and the stance of the U.S. Department of Education 
that waivers will not be granted. If the Department is too inflexible and compli- 
ance-oriented on such issues as whether states can keep aspects of their assess- 
ment and accountability systems that are working well, this could erode state 
and local support for the law and impede progress toward the broader goal of 
improving student achievement. 

B States are making headway in carrying out the law's requirements for standards 
and assessment, because most states already have experience in these areas. 

(An exception is in defining adequate yearly progress (AYP), which many states 
did not plan to resolve until final regulations were issued.) States are moving 
more slowly on implementing newer roles, such as developing lists of approved ° 0 
providers of tutoring and similar kinds of supplemental academic services, help- ° 

ing local school districts assess the qualifications of Title I instructional aides, or 1 

providing guidance to school districts about which instructional practices are 
based on scientific research. 

B Although much of the media coverage about NCLB has focused on its require- 
ments to mandate public school choice for children in under-performing schools 
and to allow federally funded supplemental education services to be provided 
by faith-based and for-profit groups, the law's greatest impact is likely to come 
from its more technical, less newsworthy provisions that require schools and dis- 
tricts to show adequate yearly progress in test score gains or else undergo 
increasingly severe corrective actions. 
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■ The Act is asking states to do more than they have ever done in education at a 
time when most states face severe economic problems. With many states look- 
ing at budget cuts and hiring freezes, state departments of education may lack 
the funding, staff, and expertise needed to fulfill all these demands. For exam- 
ple, states may not be able to provide needed technical assistance if large num- 
bers of schools are cited for inadequate yearly progress. 

■ If federal education funding for fiscal year 2003 provides only the modest 
increase proposed in the President's budget, states and school districts will be 
hard pressed to carry out the many new demands of NCLB. Without substantial 

2 new federal funding in the coming years, it will be difficult for states and school 

0 ° 0 districts to meet the ambitious goals of the Act, however worthy. 

o o 



A Broader and Stronger Federal Role 



\ \( >y hen President Bush signed the No Child Left Behind Act (Public Law 107- 
A //\\ H HO) on January 8, 2002, the federal government took on a broader and 
\J \J stronger role in education. This legislation, which passed the Congress with 
bipartisan support, extends and revamps dozens of federal programs authorized by the 
Elementary and Secondary Education Act (ESEA) and other statutes, from the $10.3 
billion Title I program for disadvantaged children to the $25 million program for char- 
acter education.The Act builds on earlier school reform legislation supported by Presi- 
dent Clinton and on the state-initiated movement to reform education by setting chal- 
lenging academic standards, but it is far more than a minor tweaking of existing law. 

Rather, it marks a significant change in federal and state responsibilities in education. 

The objectives of the law are bold and inspiring — to ensure that all children 
become proficient in reading, mathematics, and science and to close the achievement 
gap. They are also very ambitious. Implementing this 669-page Act is an immense 
undertaking at the federal, state, and local levels — and states and local school districts 
are just beginning to see how immense it will be. 

With the No Child Left Behind Act, Washington is knocking harder on the school- 
house door. The federal role is now meant to affect every public school student and 
school in the country, not just those that participate directly in federally funded pro- 
grams. The federal government is putting more pressure than ever before on states and 
school districts to raise the academic achievement of all students, especially as measured 
by test scores — and to do it on a strict timeline. Schools must close achievement gaps 
between different racial, ethnic, income, and other subgroups of children. States and 
school districts must also ensure that all teachers of academic subjects in any school are 
highly qualified according to the law’s rigorous definitions. The requirements are even 
stricter for schools and school districts that receive federal Title I funds. If these schools 
and districts do not make adequate progress toward achievement goals, they must 
change their methods and practices, and if they still do not improve they must under- 
go increasingly severe corrective actions, eventually leading to restructuring, staff 
replacements, state takeover, private management, or dissolution for those that repeat- ° 0 ° 

edly fail to improve. 

Under NCLB, states and school districts are expected to assume more responsibil- 
ities than ever before. States must expand their testing programs, analyze test data in 
new ways, provide technical assistance to under-performing school districts and 
schools, collect data about teachers’ qualifications and help teachers become better 
qualified, develop lists of approved providers of supplemental education services, and 
much more. School districts must strive to raise test scores, collect and report detailed 
test data, close achievement gaps, provide technical assistance to under-performing 
schools, recruit highly qualified teachers and paraprofessionals, provide professional 
development to instructional staff who are not yet highly qualified, and create or 
expand public school choice programs, among other duties. 
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Purpose of This Report 

This report by the Center on Education Policy describes what the federal government 
and the states have done to prepare to implement the No Child Left Behind Act since 
it was signed into law a year ago. It also discusses implementation challenges that have 
come up at the federal or state levels and identifies issues that should be watched be- 
cause they are important or potentially problematic. In addition, the report includes 
five case studies of states and school districts that seem to be doing a commendable job 
of carrying out a particular aspect of the law. 

The report is based primarily on research conducted by the Center’s staff and con- 
sultants during the summer and fall of 2002. As described in the Methodology section 
0 0°0 0 below, we reviewed states’ consolidated applications for federal grants, conducted tele- 
phone interviews with officials in 48 states and the District of Columbia and eight 
school districts, and reviewed other research on the implementation of NCLB and its 
predecessor education laws. This is the baseline-year report for a six-year study by the 
Center that will monitor the implementation of the Act until 2007, when it is slated to 
be renewed by Congress. 

As an independent organization funded primarily through foundation grants, the 
Center on Education Policy is in a unique position to study the law’s implementation. 
Other organizations that are analyzing various aspects of NCLB represent teachers, state 
administrators, parents of students in Title I programs, governors, or other constituen- 
cies, and they must be sensitive to the needs of their members. The Center is not affil- 
iated with any constituent group and aims to be objective in its research. Furthermore, 
several studies of NCLB now being conducted by other groups are focused on partic- 
ular issues or aspects of the law, such as its choice provisions. Our study, by contrast, aims 
to be comprehensive and look across a range of major issues. 



Issues Addressed in This Report 

This report focuses on four main issues: 

□ assessment and accountability (discussed in chapter 2); 

□ public school choice and supplemental education services requirements (discussed 
in chapter 3); 

□ teacher and paraprofessional quality (discussed in chapter 4); and 

□ the use of scientifically based research to improve teaching and learning (discussed 
in chapter 5). 

We chose to study these issues because they deal with the most central and con- 
troversial aspects of the new law and will require the most preparatory work at the fed- 
eral and state levels. They are also issues that must be addressed during the first year of 
implementation. 

As the nation approaches the first anniversary of the law’s signing, most of the 
action related to the law is occurring at the federal and state levels, and that is where 
we focused most of our research. The report discusses how the federal government is 
interpreting the legislation and how states are laying the foundation for its implemen- 
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tation — building testing and accountability systems, collecting data about teacher qual- 
ifications, identifying potential providers of supplemental education services, and more. 
By the end of January 2003, states should have completed one of their hardest tasks, 
defining what constitutes adequate yearly progress in raising achievement for schools 
and school districts. The decisions states make will establish performance targets for the 
next 12 years. Our general impression is that states were focusing this year on issues that 
demanded their immediate attention, leaving other items on the back burner to be 
addressed next year. 

Although a few key requirements have already taken effect at the local level, the 
impact of NCLB is just starting to trickle down to school districts. Information in this 
report about local implementation is limited, because school districts are waiting on 
states, which in turn were waiting for guidance from the federal government. In the 
next two years, however, the trickle will become a cascade, and every public school in 
the nation will have to face the full impact of the law; they will have to meet annual 
test score targets for all children or face accountability measures if they don’t. So the 
federal and state actions discussed in this report are a precursor to the real action, which 
will occur at the local level. 

As an example of how implementation is progressing during this first year in one 
school district, this chapter includes at the end a case study of the Hamilton City 
(Ohio) School District, where the No Child Left Behind Act was signed. In years to 
come, we intend to monitor local action and hope to have more information about 
local implementation. 

The long-term objective of our study is to help explore such critical questions as 
whether children are learning more and whether teachers and paraprofessionals are bet- 
ter qualified as a result of NCLB. In future reports, we will look at available evidence 
on student achievement, teacher and paraprofessional qualifications, parents’ use of the 
options for school choice and supplemental education services, and other vital issues. 
We will try to help answer such crucial questions as whether adequate funding and 
other supports have been provided to help teachers, schools, and districts meet the goals 
of the Act, whether the law is accomplishing what it intends, and whether any negative 
consequences emerge as a result of the new demands. 

Federal Implementation 

The timelines in the law to prepare for implementation are relatively short for both 
the federal government and states, especially in light of the magnitude of the task. The 
U.S. Department of Education has accomplished several important tasks in almost a 
year, although it missed a key deadline for issuing final regulations, as discussed below. 

As shown in Box 1-A, the Department has sent letters to the chief state school offi- 
cers discussing key provisions of the law; held meetings on NCLB throughout the 
country; reviewed and approved states’ consolidated applications; and carried out other 
day-to-day tasks, such as responding to queries from states and school districts. As 
required by law, the Department conducted negotiated rulemaking on two issues, stan- 
dards and assessments. The Department issued draft and final regulations as well as non- 
regulatory guidance for specific aspects of the law. 



0 

ERIC 



18 



Center on Education Policy 



From the Capital to the Classroom 



Federal Regulations and Guidance 

In general, NCLB requires the U.S. Department of Education to have final Title I reg- 
ulations in place within one year of the date of the law’s enactment (January 8, 2003), 
and this deadline was met when final Title I regulations were issued on November 26, 
2002. But NCLB required regulations to be issued within six months of enactment 
(July 8, 2002) for the very important provisions ofTitle I that dealt with standards, as- 
sessment, accountability, and district and school improvement. The law did not specify 
whether the regulations issued on the six-month schedule must be final regulations, al- 
though one could assume that was the intent, since proposed rules are published to 
elicit public comment rather than govern programs. 

6 

o On July 5, the Department issued final regulations for standards and assessments, so 

o° °o it met the deadline for two of the critical topics. But these July regulations did not 
address the two other topics — accountability and school/district improvement — that 
were supposed to be handled on a six-month schedule. Proposed rules on these topics 
were not published until August 6, and final regulations were not issued until 
November 26, later than the six-month deadline. 

With a deadline of January 31, 2003, looming for states to submit their plans for 
implementing the laws accountability requirements, many state officials we interviewed 
expressed concern about how long it was taking the Department to publish final reg- 
ulations, especially on such crucial issues as setting the timetable for adequate yearly 
progress. (The implications of this delay are discussed more fully in Chapter 2.) 
Although the Department has sometimes been late with statutory deadlines in past 
reauthorizations of education programs, the stakes are higher this time, because states 
and school districts had to begin implementing some NCLB requirements this fall and 
because the sanctions attached to the law’s accountability provisions are greater. States 
appeared to be responding to a lack of guidance by either forging ahead with their plans 
knowing that they may need to be revised, or waiting for guidance knowing that they 
will have less time to formulate a plan. 

In our interviews with state officials, we asked them to rate the quality and timeliness 
of regulations and other guidance from the U.S. Department of Education in the follow- 
ing three areas: assessment, teacher quality, and scientifically based research. For their rat- 
ings, interviewees were asked to pick one of four choices: excellent, good, fair, or poor. 

QUALITY OF GUIDANCE 

In general, states rated the quality of the Department’s guidance as good to fair, with 
guidance on assessments and teacher quality receiving higher ratings than that on sci- 
entifically based research. The specific responses for the guidance on assessment, teacher 
quality, and scientifically based research are reported in chapters 2, 4, and 5. 

This finding may reflect the fact that respondents said that there were generally 
more materials available for assessments and teacher quality than in other areas. Perhaps 
people equated quality with availability, which also could explain why scientifically 
based research, the area with the most “poor” ratings, was also the subject of many com- 
plaints about a lack of guidance. Several respondents were reluctant to answer the ques- 
tions about quality because they felt it was “unfair” to give a rating when they had not 
yet received actual guidance in many of these areas. 
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BOX 1 A CHRONOLOGY OF MAJOR ACTIONS OF THE U.S. DEPARTMENT OF 
EDUCATION ON KEY PROVISIONS OF THE NO CHILD LEFT BEHIND 
ACT ADDRESSED IN THIS REPORT 



DATE 


ACTION 


JANUARY 8, 2002 


No Child Left Behind Act signed into law 


FEBRUARY 7, 2002 


Letter on key provisions sent to chief state school officers 


MARCH 11-20, 2002 


Negotiated rulemaking conducted on standards and assessments 


APRIL 1, 2002 


Guidance published for the Reading First program 


APRIL 28, 2002 


Letter on paraprofessionals sent to chief state school officers 


MAY 6, 2002 


Notice of Proposed Rulemaking for standards and 
assessments published 


MAY 6-30, 2002 


Regional meetings on standards and assessments held in five cities 


MAY 22, 2002 


Final notice published on contents of consolidated applications 


JUNE 6, 2002 


Draft guidance published on Title II, Improving Teacher Quality 


JUNE 12, 2002 


Consolidated applications due from states 


JUNE 14, 2002 


Letter sent to chief state school officers containing guidance on 
school choice and supplemental services 


JULY 1, 2002 


Department announces 8,652 schools in need of improvement 
nationwide 


JULY 5, 2002 


Final regulations published for standards and assessments 


JULY 24, 2002 


Letter sent to chief state school officers containing guidance on 
adequate yearly progress 


AUGUST 1, 2002 


Guidance published on Comprehensive School Reform program 


AUGUST 5, 2002 


Draft guidance on supplemental education services published 


AUGUST 6, 2002 


Draft regulations published for Title 1 issues other than stan- 
dards and assessments 


OCTOBER 4, 2002 


Draft guidance published on transferability of funds 


OCTOBER 22, 2002 


Letter sent to chief state school officers on identifying schools in 
need of improvement 


NOVEMBER 7, 2002 


Draft guidance published on paraprofessionals under Title 1 


NOVEMBER 26, 2002 


Final regulations issued for Title 1 


DECEMBER 9, 2002 


Draft guidance issued on public school choice 


DECEMBER 12, 2002 


Draft guidance issued on supplemental education services 
(updated) 


DECEMBER 19, 2002 


Draft guidance issued on Title II (updated) 



Source: Center on Education Policy compiled from various sources 
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One of the main concerns states raised about the quality of written guidance was 
that it did not provide much additional information or go beyond what was in the 
statute.The regulations that have been issued did not always provide states with as much 
clarification as they wanted. At the time of our interviews, states had hoped for more 
clarification of such issues as whether a long-term substitute must meet the teacher 
qualifications of the NCLB and which commercial assessments of English proficiency 
meet the law’s requirements for testing English language learners. 

Many state officials also reported being frustrated by what they saw as a failure of 
the U.S. Department of Education officials to directly answer their specific questions 
when talking with them. Numerous state people reported to us that the career profes- 
sionals in the Department were “very cautious” in what they said about the law and 
q O O q seemed to be unable to respond to questions unless their answers had been fully cleared 
through the Department’s political appointees, which these state people viewed as a 
departure from how the Department had operated in the past. Some state respondents 
also said that the career professionals in the Department were doing their best, given the 
situation. Other state officials noted that they had gotten different answers from differ- 
ent people in the Department or had received answers that were subsequently changed. 
Although this situation has occurred before with a new law, it creates more difficulties 
with NCLB, because the stakes are higher and unlike earlier laws, there was no year-long 
transition period for states and school districts to prepare before the law took effect. 



TIMELINESS OF GUIDANCE 

When asked about the timeliness of guidance, several state officials we interviewed 
replied, “What guidance?” referring to the fact that as of late fall, they were still awaiting 
definitive guidance on several major issues. Average ratings of the timeliness of the De- 
partment’s written guidance were in the fair range, with the written guidance on issues 
of assessment and teacher quality again rating higher than the guidance on scientifically 
based research. The specific ratings for these issues are reported in chapters 2, 4, and 5. 

In the Department’s defense, it can be said that federal officials have had to grapple 
with some complex issues that are not easy to interpret, beyond what is in the statute. Most 
importandy, the timelines in the law itself are short, in light of what the law directs states 
and the federal government to accomplish during the first several months. Some states 
interviewed noted that they wished the law had authorized a year to plan for transition, to 
allow for better, more thoughtful preparation at the local, state, and federal levels. 




Flexible or Stringent? 

The No Child Left Behind Act is often described as a law that brings greater flexibility 
to federal education programs in exchange for greater accountability for results. Several 
provisions of the Act allow states and school districts greater latitude or decision-mak- 
ing authority in the use of federal funds, such as the “Ed Flex” program, the various 
options for waivers of federal requirements that are impeding progress toward the laws 
broader goals, and the new “transferability” provision that allows states and school dis- 
tricts to shift funds among various federal program authorities. At the same time, how- 
ever, these more flexible options are counterbalanced by the numerous detailed re- 
quirements in other sections of the law. Therefore, a key question is how much 
flexibility will be allowed in the implementation of NCLB. 
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Not surprisingly, many state officials that we interviewed wanted to see the U.S. 
Department of Education show some flexibility in interpreting the law, so the states 
could build on their successful existing practices, meld federal efforts with state reforms, 
and address circumstances peculiar to their individual states and districts. Moreover, 
many of the statutory requirements are extraordinarily specific; a bit of flexibility could 
help states and school districts focus their energies on the most important goals of the 
law, rather than being continually anxious about compliance. A statement issued by the 
Council of Chief State School Officers on November 10, 2002, reiterated the states’ 
desire for flexibility. “The President has consistently stated his support for local control, 
and it seems logical that a ‘one size fits all’ compliance system regarding NCLB would 
be counterproductive,” the statement read. “State leaders are concerned that any of their 
suggestions to use current state practices as a means for implementing NCLB will be 
viewed as an attempt to either discredit the legislation or ‘water down’ its intent.” 

Some earlier communications from USED were encouraging, in that they suggest- 
ed that some flexibility might be forthcoming. In a letter to the states of July 24, 2002, 
Secretary Paige noted that the intent of the law “is to build on high quality accounta- 
bility systems that States already have in place, not to require every State to start from 
scratch.” And in the draft regulations published in August, the Department invited 
“states that have been using different models to comment on... how these requirements 
could be incorporated into their current systems.” 

In later statements, however, and in Title I final regulations, the Department took a 
tighter approach. In a speech on October 24, 2002, Susan Neuman, Assistant Secretary for 
Elementary and Secondary Education, stated, “The previous administration was waiving 
this and waiving that. This administration is serious. We don’t intend to waive any of the 
requirements.” (Balta, 2002). 

The final Title I regulations adopt a strict interpretation of the statute in many 
areas, requiring, for example, that school districts offer public school choice even if the 
potential receiving schools are filled to capacity and requiring states to use the same 
definition of steady yearly progress for all subgroups of students. Since the statute itself 
is specific and detailed, the Department may have felt that allowing flexibility would be 
inconsistent with the law’s intent. On the issue of adequate yearly progress, for exam- 
ple, the law clearly contemplates an aggressive schedule for school improvement, even 
if it means that in any given year, a large number of schools will be cited for improve- 
ment. Department officials may be afraid that if they issued regulations that gave states 
more latitude in this area, it could be viewed as weakening the law. 

While a policy of liberally granting waivers could undercut the intent of the law, 
so too could a “strictly by the book” approach to compliance. Granting no waivers 
could be counterproductive to achieving the law’s purposes and could result in less 
effective action in states. Flexibility that recognizes local and state context and reform 
efforts may enhance reform. For example, in Texas, several waivers were granted under 
the Ed-Flex authority, which may have helped to produce test score gains in Texas. 
Waivers need not and indeed cannot legally waive the intent and purposes of the law. 
But when flexibility is carefully considered and permitted on a case-by-case basis, it can 
sometimes help to achieve the overall goals of the law. 

On a related topic, the state officials we interviewed rarely mentioned the flexibil- 
ity provisions of NCLB, such as Ed-Flex and transferability, as being critical to educa- 
tion reform. We do not know why this is the case; perhaps they view the flexibility 
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options as being limited, or perhaps the federal officials have not adequately explained 
the options available and how to use them. It is possible that states and school districts 
are overlooking the opportunities afforded by the flexibility provisions to think cre- 
atively across programs, and that these provisions will come into play in future years as 
their potential becomes more obvious. 




Federal Funding 

A persistent question during the first year of implementation is whether the federal 
government will provide enough funding for states and school districts to accomplish 
the ambitious goals of NCLB. Several state officials we interviewed estimated the full 
costs of implementing the law to be enormous and felt that the federal government 
had a responsibility to provide adequate additional funding to help states and school 
districts shoulder all the new responsibilities. Furthermore, a notable share of funding 
must go toward specific aspects of the legislation, such as professional development or 
choice and supplemental services. 

The amount of federal funding will have a tremendous impact on implementation 
in the coming years. As this report went to press, the fiscal year 2003 appropriations for 
education had not yet been finalized. If education receives only the modest 2.8% 
increase proposed in the Presidents budget, this amount is likely to fall far short of what 
will be needed for states and school districts to effectively implement the new law. If 
state and local governments perceive that the federal government is not covering an 
adequate share of the costs, their commitment to the law is likely to wane, no matter 
how worthy its goals. Even if providing a high-quality education for all is a primary 
responsibility of state and local governments, they may not be able to readily fulfill this 
responsibility in the current economic climate. 



State Implementation 

Since the No Child Left Behind Act was signed into law, most states seem to be mak- 
ing reasonable progress with implementation on such fronts as developing state appli- 
cations, revising assessment systems, and developing or refining systems to track the 
qualifications of teachers. Although changing state assessment systems is a major task, 
states seem to have made the most progress with the standards and assessments require- 
ments, because these are areas they have had the most experience with. They seem to 
have made less progress with newer roles, such as developing lists of approved non- 
school providers of supplemental education services, helping local school districts assess 
the qualifications ofTitle I paraprofessionals, or providing guidance to local districts 
about which instructional practices are based on scientific research. 



State Support for the Goals of NCLB 

Most state officials we interviewed expressed strong support for the intent of NCLB 
and said that the goal of improving the achievement of all students is “the right one.” A 
few interviewed admitted that they thought the goals were commendable, but they 
were doubtful about whether they could be accomplished in the 12-year time frame 
envisioned by the law. 
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Two particular provisions of the law met with wide approval among state officials. 
First, state officials saw the emphasis on providing highly qualified teachers as essential 
to improving the performance of students in low-achieving schools. Second, state offi- 
cials, like the President and the laws Congressional sponsors, hoped that the new 
requirements to disaggregate achievement data for various subgroups of students would 
bring greater attention to the needs of students who traditionally underachieve and 
would better focus resources on closing achievement gaps. 

Many states also expressed their approval of the continued emphasis on standards 
and assessments in NCLB. Several states said that the federal law is in many ways an 
extension of the reform that was already taking place in their states. Putting into place 
the new assessments required by NCLB was not a major concern of states. Most states 
already had standards in place or were far along in developing them. States also had rel- 
atively long-term relationships with test developers and had begun planning for new 
assessments or augmentations for old assessments. 

A few state officials mentioned in our interviews that NCLB was having a positive 
impact in their states because state people outside of the state department’s federal pro- 
gram office were beginning to attend to federal programs. One respondent said this was 
the first time she had noticed “the state superintendent becoming involved in how fed- 
eral legislation plays out in the state. That has never happened in the past.” 



Progress on State Applications 

The first major step for states in implementing the No Child Left Behind Act was to 
develop and submit a consolidated state application for the major federal formula grant 
programs to the U.S. Department of Education by June 12, 2002. This application in- 
cludes the state plan for carrying out all of the programs of the Elementary and Sec- 
ondary Education Act. All 50 states and the District of Columbia have developed their 
applications and met this deadline. 

The consolidated application must include a statement that the state accepts a list of 
five specific performance goals for ESEA generated by the U.S. Department of Education 
and twelve specific indicators of progress toward those goals. The five goals, shown in Box o q G 0 
1-B, relate to improvements in student achievement, proficiency for English language ° 

learners, teacher qualifications, safe and drug-free learning environments, and high school 1 1 

graduation. States could add other state-specific goals to their statements. 

In future years, states will have to submit an annual performance report to ED that 
includes data demonstrating their progress towards the federal and any state goals. The 
consolidated applications must also describe the activities and programs states are 
implementing to fulfill these goals. Finally, the applications must include assurances 
about key ESEA program requirements and fiscal information. 

The state applications vary considerably in length and depth. By the time the 
Center had completed its review, all state applications had been approved conditional- 
ly by the Department, meaning that they are approved as long as states address certain 
conditions or omissions specific to that state. The two conditions raised most often by 
the Department pertained to the “unsafe school choice option” in NCLB and services 
for migrant children. As a condition for full approval, all 50 states and the District of 
Columbia were told to be more explicit in their plans about how they will implement 
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the new “unsafe school” choice requirements, which gives students the option of trans- 
ferring to a safe public school if their own school has been labeled “persistently dan- 
gerous” or if they become victims of violent crimes at their school. About half the states 
were also asked to include more information in their plans about how they will meet 
the special educational needs of migrant children. 



State Capacity 

States will need expertise and personnel to fulfill such NCLB duties as providing tech- 
nical assistance to persistently failing schools and districts — especially if that number 
greatly increases. They will need adequate staff, funds, and expertise to develop and ad- 
minister assessments, collect and analyze data on student achievement and teacher qual- 
ifications, determine which providers of supplemental services are acceptable, and carry 
out a host of other responsibilities assigned them under the new law. 

One of the concerns that state officials raised most often in our interviews was 
whether their state government had the capacity to effectively implement the new law, 
both in the near future and down the road.These additional demands are coming at a time 
when the majority of states are facing the most ominous fiscal crisis since World War II 
(National Governors Association, 2002) .Various factors have caused or contributed to this 
situation, including an explosion in health care costs, the loss of revenues from the falling 
stock market and the economic slowdown, state tax cuts enacted in prior years, and other 



BOX 1 -B ESEA PERFORMANCE GOALS IN THE 
CONSOLIDATED STATE APPLICATION 



PERFORMANCE GOAL 1: By 2013-2014, all students will reach high standards, at a mini- 
mum attaining proficiency or better in reading/language arts 
and mathematics. 



PERFORMANCE GOAL 2: All limited English proficient students will become proficient in 
English and reach high academic standards, at a minimum at- 
taining proficiency or better in reading/language arts and math- 
ematics. 



PERFORMANCE GOAL 3: By 2005-2006, all students will be taught by highly qualified 
teachers. 



All students will be educated in learning environments that are 
safe, drug free, and conducive to learning. 



PERFORMANCE GOAL 4: 

PERFORMANCE GOAL 5: All students will graduate from high school. 



Source: U.S. Department of Education, Consolidated State Application 




factors. Even after making significant spending cuts, 37 states were forced to reduce their 
enacted budgets for fiscal year 2002, and 15 states have already laid off employees. 

NCLB authorizes funds for state administration, mostly through provisions that set 
aside a small percentage of federal formula grant funds for state administration. This is 
the case in the Title 1 program for disadvantaged children, the Title II program for 
teacher quality, and several others. But some state officials we talked to feared that these 
administrative allotments would not be enough to fulfill all the new responsibilities 
associated with the federal law, no matter how worthy the NCLB goals. 

As a result of budget cuts made to reduce funding deficits, weather economic 
downturns, or shrink the size of state government, several officials we interviewed said 
that the state staff for many federal programs is already down to the bare minimum, and 
in some cases one person is responsible for implementing multiple state and federal pro- 
grams. Also, at least one state noted that federal funds were being stretched to cover the 
salaries of state department of education staff who are only indirectly involved in the 
federal program administration, such as supervisors of those responsible for the imple- 
mentation of federal programs. 

In one large state that we interviewed, the state department of education has been 
subject to budget cuts, hiring freezes, elimination of positions, and early retirement incen- 
tives since the mid-1990s. As a result, the department staff has been cut by roughly half 
and has become increasingly reliant on federal funds to support its operations: one offi- 
cial we spoke with estimated that 60 to 65 percent of that state department’s staff were 
supported by federal funds. This official also said that since the early 1990s, the state staff 
has evolved from one made up largely of subject matter specialists (such as reading spe- 
cialists and bilingual education experts) to one composed mostly of generalists who can 
better assist the field; but the lack of subject matter expertise has sometimes hampered the 
state s effectiveness in carrying out federal programs. The official also said that for the state 
to effectively implement NCLB, it would need an additional 25 to 30 new employees. 

In Vermont, Governor Howard Dean proposed the idea that the state should not 
accept federal Elementary and Secondary Education Act funds because the costs to the 
state of complying with the law were too great (Keller, 2002). Ultimately, the state did 
accept the federal funds, but Governor Dean is by no means alone in his concern about O q Q 0 
the costs of compliance. As the pressures from strong accountability measures escalate, 0 

the view could become more widespread that that the federal government is demand- 13 

ing too much, too fast, in light of its limited contributions. 



State and Local Control 

States with a strong history or philosophy of local control raised concerns about 
NCLB’s impact on local and state decision-making.This concern is not new to federal 
programs, but some control issues specific to NCLB will have to be addressed. First, in 
a few states, several of the accountability provisions of NCLB are not allowed under 
state law, such as state takeover of schools or charter schools; this limits a state s options 
in dealing with schools in the later stages of school improvement. These states said that 
they would have a difficult time convincing their state legislatures of the need to enact 
changes in state law to accommodate NCLB. Second, states were concerned that the 
scientifically based research requirements of the law would essentially evolve into a list 
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of accepted programs that districts would have to choose from, limiting their ability to 
make programmatic decisions. This issue is discussed in more depth in chapter 5. 



Methodology 

To study implementation of NCLB, the Center on Education Policy used a variety of 
research methods. 



14 Review of State Consolidated Applications 

q O C> 0 Between July 30 and September 10, 2002, a Center research associate and a consultant 
reviewed 51 consolidated applications submitted to the U.S. Department of Education 
by all 50 states and the District of Columbia. The reviewers used a protocol that con- 
sisted of 15 questions on the following issues: state application approval, standards, as- 
sessments, adequate yearly progress indicators, statewide system of support, educator 
quality, allocation of school improvement funds, cost of developing assessments, supple- 
mental education services, public school choice, and assessments for English language 
learners. (The Protocol for Review of State Applications can be found in Appendix 1.) 
In order to capture correspondence that took place between the U.S. Department of 
Education and state education administrators after the initial applications were submit- 
ted, the two reviewers also obtained access to the USED Consolidated Application files, 
which included associated correspondence between USED program specialists and the 
states, and addendums providing additional information that was requested by USED 
or not submitted with the original consolidated application. 



State Interviews 

During September, October, and November 2002, several Center staff and consultants 
conducted telephone interviews with 48 states and the District of Columbia. Inter- 
view questions were based on a guide developed by the Center that included 33 mul- 
ti-part questions about assessment and accountability, teacher quality, requirements for 
scientifically based research, and the state’s general views about implementation of the 
law. On average, we spoke with three individuals per state, most commonly with state 
Title I directors, individuals overseeing the state assessments, and officials from the state 
teacher certification/licensure division. Each state interview took an average of two 
hours to complete. Those interviewed were given an opportunity to review their re- 
sponses, and revisions were made to the original survey answers as necessitated by those 
reviews. The interview guide is included in Appendix 2. Not all the states answered 
every question in our survey, and the interviewers in some cases departed from the 
guide to explore topics of special interest in a given state or follow up on a potentially 
useful line of discussion. 




27 



Case Studies 

To gather information for case studies, several Center staff and one consultant con- 
ducted in-depth interviews in November 2002, with personnel from two states and 
three school districts about specific provisions of NCLB. These states and districts were 
identified through the state interview process described above and through conversa- 
tions with experts in the field. Interview protocols were developed for each site and 
topic area. The interviews were structured to be as open-ended as possible but includ- 
ed specific probes based on general findings from the state-level interviews in the ap- 
propriate content area. Interviewees were given an opportunity to respond to the case 
studies, and revisions were made as needed. 



Research Literature Reviews 

Center staff and consultants also conducted background research on the various topics 
addressed by the report and have noted the sources of this information throughout the 
report. 



Attributing Information to Individuals 

In this report, we generally have not attributed statements or actions to specific indi- 
viduals or states by name, unless the information could be found from other sources 
such as documents on state websites or newspaper articles. In order to get candid an- 
swers from the state officials we spoke with, we thought it was important to afford them 
anonymity. Our concern was that state officials would be reluctant to talk “on the 
record” about how their state was proceeding with the early stages of implementation 
of the No Child Left Behind Act, for fear of reprisal from the U.S. Department of Ed- 
ucation if they were not in complete compliance with the law. For example, at the time 
of our interviews, many states had not approved supplemental education service 
providers, and some were reluctant to report this on the record. 



Conclusion 

The No Child Left Behind Act embraces a compelling set of goals that goes to the 
heart of the educational process. Past federal laws have tried to improve students’ learn- 
ing, but none has addressed this goal more directly or forcefully than NCLB, which 
challenges U.S. schools to raise the achievement of all students to proficient levels. Im- 
plementing this ambitious law will require renewed dedication of educators, state offi- 
cials, and federal administrators. It will take the cooperation of national and state lead- 
ers to provide the assistance needed to help local staff do their jobs. The coming years 
will show whether all levels of government will rise to the challenge of making public 
schools better for all children. 
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CASE STUDY 

HAMILTON, OHIO: ONE YEAR LATER 
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While Hamilton City School District agrees with NCLB's goals, it may take a 
little more help to smooth out some of the bumps along the way to full 
implementation. 

On January 8, 2002, President Bush signed the No Child Left Behind Act at Hamilton 
High School in Hamilton, Ohio. Almost one year later, staff from the Center on Edu- 
cation Policy spoke about implementation with people in the Hamilton City School 
District, where officials said that they “feel a moral obligation to put this [NCLB] to- 
gether and do a good job.” 

Officials in the district agree with the goals of the Act. They especially like the focus 
on long-term, comprehensive planning, which the district has been engaged in for sever- 
al years; being able to coordinate grants and goals across programs; the emphasis of read- 
ing, math, and science; and the additional funds for English language acquisition programs 
under Tide III. They are also encouraged by the heightened federal interest in education 
signified by this Act and believe that “the current Congress is serious about wanting to 
improve education” While they have some concerns about meeting the requirements of 
the new law, as one official said, “You have to take some challenges, but they are worth it.” 
At the same time, the district would like additional guidance and support from fed- 
eral and state officials in helping to implement the Act. However, with the exception 
of a few hurdles, implementation has been smooth in the district. 



School Improvement 

Out of ten Title I schools, Hamilton had two schools in the first year of school im- 
provement and was required to offer school choice to parents in those schools. No 
schools were required to offer supplemental services. Schools were only chosen as trans- 
fer schools if their student achievement scores had met the baseline performance level 
that Ohio plans on setting as part of its definition of adequate yearly progress. 

Approximately 13 students from each building, out of a total of 650 students, took 
advantage of the school choice option and transferred to higher-performing public schools. 
Given the small numbers of transfers, all students received placements either in their par- 
ents’ first or second choice school. While applications did not ask for the reason why par- 
ents were requesting a transfer, district officials asked them informally. Approximately half 
said they transferred for academic reasons and half for other reasons such as child care. 

Delays in receiving student achievement data and setting AYP levels caused signif- 
icant problems in identifying schools that had not met AYP in some Ohio districts and 
almost did in Hamilton. The district considered this the only real “implementation 
snag” that it ran into. An early list released by the state of schools in need of improve- 
ment identified several hundred more schools than the final list did, and some districts 
in the state sent out letters to parents informing them of choice and supplemental serv- 
ices options that were not eventually offered. Hamilton originally had three schools on 
this state list rather than two and would have been required to offer supplemental serv- 
ices if the original list had held. This incident points out the need for both speed and 
accuracy of these lists to help districts plan for these services. 

Another concern about identifying schools for school improvement is the difficul- 
ty in getting out of school improvement, even if the school is making significant head- 





way. One school currently labeled in need of improvement is outperforming some of 
the other schools in the district and has seen enormous improvement in recent years. 
In March 1996, 8.8% of this schools students were proficient on the state reading test; 
in March 2002, 52% were proficient, a higher percentage than some other schools in 
the district. Officials would like the accountability system to do a better job of recog- 
nizing this kind of growth. 



Teachers and Paraprofessionals 

When staff from the Center on Education Policy spoke to Hamilton officials, Ohio 
did not yet have guidelines on the highly qualified teacher provisions of NCLB, so the 
district had not yet sent out letters to the parents informing them about their teachers’ 
qualifications. While the district is very concerned about the compliance ramifications 
of this, officials from both USED and the Ohio Department of Education have told 
district personnel that they will be considered in “delayed compliance” until the state 
puts guidelines together. 

During a series of budget cuts in 1980, Hamilton phased out instructional aides in its 
Title I program and currently only uses aides for special education and a few other pro- 
grams, so the district does not anticipate significant problems in meeting the NCLB 
requirements for qualifications of paraprofessionals. To help the district gauge the qualifi- 
cations of its paraprofessionals, officials actively sought to be part of the states piloting of 
the ParaPro assessment so that it would have some preliminary information on its staff. 



Budgetary and Capacity Issues 

The law newly requires districts to set aside a portion ofTitle I funds for school choice. 

This reprogramming of funds affected both staffing and the number of schools receiv- 
ing Title I funds. While no staff members were fired, several Title 1 positions were not 
refilled due to budget constraints. In addition, this year the district had two new schools 
that were eligible for Title I funds. In the past, the district tried to serve all eligible 
schools, but they did not add these two schools this year. 

Although the district is not afraid to meet the challenges of the new law, it would like 
more support and guidance to help it meet the requirements. This means not only time- 
ly guidance on issues like teacher quality and AYP, as mentioned above, but also qualified, °o° 

knowledgeable support staff at the federal and state levels. The district is concerned that 17 

technical assistance providers do not have enough background in federal or state programs 
to help it implement the Act. This may be a result of budget cuts that make it difficult to 
“find quality people,” but the district is not finding the expertise it needs. 

The district would like help in identifying programs that fit well with the states 
assessment system and will help them improve student achievement. One school in the 
district implemented a comprehensive school reform model “by the book,” yet it is one 
of the schools in school improvement. The district studied why the school showed no 
improvement and found that the models emphasis was so different from the state 
assessment system’s emphasis that there was little apparent improvement as measured by 
state tests. Outside support that could have informed the district in advance of this kind 
of mismatch would have been of great help to the district. 



Source: Center on Education Policy school district interviews 
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CHAPTER 2 




Student 




KEY FUNDINGS 

M There is broad support among states for the ultimate goal of the No Child Left 
Behind Act — to ensure that all subgroups of students are performing at the pro- 
ficient level in the near future. 

□ States are making a good faith effort to comply with the new law, while attempt- 
ing to preserve aspects of their existing testing and accountability systems that 
are working well. States are making progress in carrying out the law's require- 
ments for standards and assessments, possibly because these are areas in which 
states already have considerable experience. 

□ Virtually all state officials interviewed agreed that the most formidable challenge 
of the law is to come up with a suitable plan for measuring adequate yearly 
progress. State efforts to do this have been hampered by the federal govern- 
ment's delay in issuing final regulations on complex questions and controversial 
issues related to AYP. 

□ The rules for determining whether schools and districts are making adequate 

yearly progress present practical and technical dilemmas and could result in 
large numbers of schools being identified for improvement. States are con- 
cerned that if they have to assist all of the schools identified for improvement, 
this will spread their resources too thinly and subvert the law's intent of target- 
ing extra money and help on the schools and students that need it most. °o° 

O Some states have more rigorous standards, assessments, and definitions of pro- 19 

ficiency than others, which affects the percentages of students attaining profi- 
ciency in each state. NCLB continues to allow each state to define proficiency in 
its own way, which means that states that set higher expectations could be unin- 
tentionally penalized, because their schools will have farther to go to reach the 
goal of 100% proficiency, while states with lower expectations could have an 
incentive to keep them low. 

n The federal government should closely monitor how the rules for implementing 
AYP play out in different states in the coming years, allow exceptions to states 
as needed to achieve the broad intent of the law, and be open to revising the 
specific assessment and AYP requirements as evidence of their consequences 
accumulates. 
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sing tests to evaluate whether schools are making adequate yearly progress in 
student achievement is nothing new for some states. What has changed with 
W 7 enactment of the No Child Left Behind Act is that states now have far less 
latitude to determine who gets tested, how and when they get tested, and what consti- 
tutes AYP.The scope of the federal assessment and accountability requirements — so 
called because they are meant to hold schools accountable for raising student achieve- 
ment — has also expanded to affect all students in every school, not just those in schools 
supported by the federal Title I program for disadvantaged children. 



Requirements of Prior Law 

Under the 1994 Improving America s Schools Act (IASA), which reauthorized most of 
the federal elementary and secondary education programs, states that received funds 
under Title 1 of that Act (which all states did) had to assess the progress of students in 
Title I-supported schools in both reading/language arts and mathematics at least once 
in the elementary, middle, and secondary school grade spans, using assessments aligned 
with state academic standards. The standards and assessments used to measure progress 
of children in Title I schools had to be the same as those used for all children in the 
state, if the state had a general system of standards and aligned assessments. But the def- 
initions of AYP and the interventions that followed for low-performing schools were 
left to state determination and, in some states, applied only to Title I schools. More- 
over, enforcement was weak. Only 17 of the states and jurisdictions were in compli- 
ance with the requirements for final assessment systems by 2001; 35 were put on time- 
line waivers or compliance agreements. (This past inaction may explain why the 
Department is so reluctant to grant waivers under No Child Left Behind.) 



Assessment Requirements of NCLB °o c 

The No Child Left Behind Act has substantially increased both the frequency and scope 21 

of the testing requirements. Each state must develop a single accountability system for 
all schools that is based on challenging academic standards and aligned assessments, and 
includes sanctions and rewards tied to school performance. By the 2005-06 school year, 
states must begin administering these assessments in reading/language arts and mathe- 
matics every year to all students in grades 3 through 8 and once during high school. In 
addition, beginning with the 2007-08 school year, states must give a science assessment 
annually in at least one grade at the elementary, middle, and high school levels. 

Most states do not have assessments in all of these grades or subjects, which means 
that they have to make revisions and additions to their state testing systems. As illus- 
trated in Figure 2-A, data from Education Week’s annual state survey for 2002 show that: 
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IS Nineteen states and the District of Columbia administer both reading/language arts 
and math assessments in grades 3-8 plus one assessment in high school, as required by 
NCLB. 

0 Twenty-five states administer science assessments in at least one grade at the ele- 
mentary, middle school, and secondary school levels, as will be required by NCLB 
in 5 years. 

m Only six states appear to administer all the required assessments in all of the sub- 
ject areas and grade levels, if one only counts those states using tests aligned with 
state standards. 

Adequate Yearly Progress Requirements 

The NCLB requirements for measuring adequate yearly progress are also substantially 
more demanding and broadly applied than under prior law. The 1994 amendments 
gave states considerable leeway in how they defined adequate performance for Title I 
schools. Michael Cohen (2002), a former USED official in the Clinton Administration, 
reflects that under the previous legislation: 

0 About a dozen states expected from 90% to 100% of students in each school to meet 
the states definition of proficient performance, while another 10 states set a goal of 
50% at the proficient level in order for a school’s performance to be satisfactory. 

m Only 14 states set specific timelines for schools to meet performance goals (on 
average 10 years, with a range of 6 to 20 years). 

0 States used vastly different methods for defining AYP. Some required schools to 
meet an absolute performance target, while others required relative improvement 
each year or reductions in achievement gaps between subgroups of students, and 
yet others used various combinations of these approaches. 

States varied tremendously in the proportion ofTitle I schools identified as need- 
ing improvement. At the low end, Arkansas and Wyoming did not identify any of 
their schools for improvement, Texas identified only 1%, and North Carolina iden- 
tified less than 5%. At the other extreme, Michigan identified 76% ofTitle I schools 
as needing improvement, and Washington, D.C. identified 80%. These contrasts 
reflected differences in states’ definitions of AYP and in the rigor and types of tests 
used, more than actual differences in performance between the states. 

The adequate yearly progress provisions of NCLB are much more prescriptive and 
apply to all schools and school districts in the state. The new law specifies that each 
.state’s AYP plan must lead to the goal of all students reaching proficient levels on the 
state assessments within 12 years (by the 2013-14 school year). States will still deter- 
mine what constitutes “proficient” performance on their tests. This school year, states 
must set a “starting point” (also referred to as the initial bar) for each subject area, rep- 
resenting the minimum percent of students who must score at or above the proficient 
level. States are to use the higher of two options: 1) the percentage of students at the 
proficient level for the lowest-performing subgroup of students statewide; or 2) the 
point at which one-fifth of all students are in schools with lower proficiency levels. 
Experts predict that most states will use the second option, because that figure will 
almost always be higher than the proficiency level for the state’s lowest-performing sub- 
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FIGURE 2-A CURRENT STATE TESTING POLICIES COMPARED 
WITH NO CHILD LEFT BEHIND REQUIREMENTS 




□ 



States that currently 
test students in English/ 
language arts and mathematics 
at grades 3-8, plus once in the 
10-12 grade span: 

Alabama, Alaska, Arizona, 
California, Delaware, District 
of Columbia, Florida, 
Georgia, Idaho, Louisiana, 
Maryland, Mississippi, New 
Mexico, North Carolina, 
South Carolina, South 
Dakota, Tennessee, Texas, 
Utah, West Virginia 



□ 



States that currently test 
students in science at 
least once in each of the grade 
spans 3-5, 6-9, and 10-12 

Alabama, Arkansas, 
Delaware, Florida, Georgia, 
Illinois, Kansas, Kentucky, 
Louisiana, Maine, Michigan, 
Missouri, Montana, Nevada, 
New Mexico, New York, 
Oklahoma, South Dakota, 
Tennessee, Utah, Vermont, 
Virginia, Washington, West 
Virginia, Wisconsin 



□ States that currently test 
students in all subjects 
and grade levels required by 
NCLB* 

Delaware, Florida, Georgia, 
South Dakota, Tennessee, 
Utah 



| [ All other states 



Total includes only those states with standards-based assessments 



Source: Education Week Annual State Survey, 2002. Used with permission of authors. 
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group (Olson, 2002, April 3). Thus, by definition, some schools in a state will already 
have scored below the target set for the baseline year.The state must raise its bar in equal 
increments, with the first increase within two years and at least every third year after 
that, so that in 12 years, all students score at or above the proficient level. 

According to regulations issued by the U.S. Department of Education in draft form 
on August 6, 2002 and in final form on November 26, the starting point must be the 
same for all schools, districts, and subgroups within a state. The regulations allow states 
to set different starting points for each grade span. This is allowed because high schools 
(and middle schools, to a lesser extent) tend to have lower percentages of students at 
^ the proficient level, and these schools are larger than elementary schools.Therefore, if a 

single starting point is set for all schools regardless of grade span, there will be a dis— 
0 0° proportionate number of high schools scoring below the starting point and, converse- 
ly, a disproportionate number of elementary schools above it. 

Under NCLB, average student test scores in reading/language arts and math will be 
the main indicator of whether schools and districts have made adequate yearly progress, 
but graduation rates and at least one additional indicator for elementary and middle 
schools, to be chosen by the state, must also be included. However, these additional indi- 
cators may not be used to reduce the number of schools or change the schools identified 
using assessments; they may only be used to increase the number of schools. 

The most challenging requirement of the new law is for schools and school dis- 
tricts to make adequate yearly progress in reading and math for each of a number of 
student subgroups, including low-income students, limited English proficient students, 
students with disabilities, and students in major racial and ethnic subgroups. At least 
95% of students in each subgroup must be tested. Any school or district that fails to 
make AYP in either reading or math for any subgroup for two consecutive years will be 
identified as in need of improvement (provided the number of students in the subgroup 
is sufficiently large to yield a valid result). If a subgroup does not meet the bar, a “safe 
harbor” provision permits schools to be considered as making AYP if they meet two 
requirements: 1) the percentage of students in the subgroup who did not achieve pro- 
ficiency dropped by at least 10% from the previous year; and 2) the school makes its 
target for improved graduation rates (for secondary schools) or shows improvement on 
the additional, state-determined indicator (for elementary schools). 

Schools and school districts that receive Title I funds but do not make adequate year- 
ly progress for two or more consecutive years will be subject to improvement strategies 
and corrective actions by the district or state, such as revising instruction, receiving tech- 
nical assistance, providing public school choice, offering students supplemental education 
services, reorganizing or replacing staff, or restructuring the school.The corrective actions 
become more severe for schools or districts that fail to show improvement over multiple 
consecutive years. States may also choose to apply the same process of improvement 
strategies and corrective actions to non-Title I schools. In the consolidated applications 
that states must submit to receive federal education funds, states cited school improvement 
and support teams as the most common strategy they planned to use to help schools and 
districts in need of improvement, with 20 states citing this option. 

The law also requires states to establish a program of recognition and reward for 
schools and districts (both Title I and non-Title I) that exceed their targets for adequate 
yearly progress or make significant inroads in closing achievement gaps between vari- 
ous subgroups of students. 
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Assessment Requirements for English Language Learners 

The Act also contains special provisions for assessing students with limited English pro- 
ficiency, also known as English language learners (ELLs).This is a large group of stu- 
dents — more than 4.7 million ELLs were enrolled in K-12 schools in 2001-02 — and 
the numbers are growing rapidly (National Clearinghouse for English Language Ac- 
quisition, 2002). NCLB requires that all school districts measure the proficiency level 
of these students in the domains of speaking, listening, reading, and writing in English. 
Assessments for these students should also measure their progress in meeting annual 
goals established specifically for ELLs, as well as their progress toward the state s aca- 
demic standards for all students. To fulfill these requirements, a state can either select an 
assessment that has been approved by the U.S. Department of Education and require all 
districts to use it, or provide school districts with the list of Department-approved as- 
sessments and allow districts to choose one. 

School districts must make biennial evaluation reports that describe the programs 
and activities conducted for ELLs during the two immediately preceding fiscal years; the 
progress made by ELLs in learning English and meeting challenging state academic stan- 
dards; the number and percentage of children attaining English proficiency at the end of 
each school year; and the progress made by students in meeting state academic standards 
for the two years after they are no longer receiving English language services. 



Federal Actions 

The No Child Left Behind Act sets out timetables and some specific actions that the 
U.S. Department of Education must follow in implementing Title I.The Department 
took several actions relevant to assessment and accountability issues. 

The Department was required to use a process of negotiated rulemaking to devel- 
op the regulations governing standards and assessments, and could also use this process 
for other topics, at its discretion. Negotiated rulemaking involves bringing together a 
group of representatives of major stakeholders in a formal setting to negotiate the text 
of regulations, within limits allowed by the statute. The Department decided to limit 
the items to be included in negotiated rulemaking to the two mandatory topics includ- 
ed in the Act, standards and assessments. This took the most contentious issues related 
to accountability and school improvement, especially adequate yearly progress, off the 
table. Draft regulations for standards and assessments, based on negotiated rulemaking, 
were issued on May 6, and final regulations were published on July 5. 

On May 22, the Department issued final regulations about the consolidated appli- 
cations that states had to submit to receive federal education funds. The regulations 
acknowledged the difficulty states faced in immediately submitting all of the informa- 
tion required by the Act, while recognizing that states had to submit applications with- 
out delay in order to receive funding in July 2002. Of particular importance was the 
timeline set for states to submit information on how they would measure adequate 
yearly progress and determine their starting points. States were given until January 31, 
2003, to submit their plans for handling these issues to USED. 
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On July 24, the Secretary sent a key policy letter to the chief state school officers 
with guidance on some issues related to Title I accountability and school improvement. 
The letter adhered quite closely to the statute, but indicated that the Department might 
allow for some flexibility in implementation. 

In late August, the Department conducted a pilot review of the accountability sys- 
tems of five states — Colorado, Indiana, Louisiana, Massachusetts, and North Carolina. As 
this report went to press, the Department had yet to provide feedback to the five states. 

In an October 22 letter to the states, Secretary Paige exhorted states to not lower 
performance standards. Some states had indicated that they intended to do so to avoid 
26 identifying very large numbers of schools as in need of improvement or corrective 

o C» 0 action or to establish a more realistic goal (as discussed in more detail in the Issues to 

Watch section below). 

As mentioned in Chapter 1, the Department published proposed regulations on 
accountability and school improvement on August 6, nearly seven months after the law’s 
enactment, and issued final regulations on November 26, almost eleven months after 
enactment — later than the six-month deadline in the law for regulations on these top- 
ics and just two months before states had to submit their AYP plans to the Department. 

In our interviews with state officials, we asked them to rate the quality of the reg- 
ulations and written guidance provided by USED on assessment issues. The majority of 
the 40 states responding to this question rated it as “good” (60%). Another 30% rated it 
as “fair”. When asked about the timeliness of this written guidance, 29% of the 41 states 
responding rated it good and 41% called it “fair”. The totals are shown in Figures 2-B 
and 2-C. 

We also asked state officials to rate the quality of other non-written forms of fed- 
eral guidance about assessment, such as oral communications, websites, and meetings. 
Five percent of the 37 states who responded to our question rated this type of guid- 
ance excellent, and 38% rated it good, while 41% thought it was fair and 16% thought 
it was poor. These ratings are shown in Figure 2-D. For the timeliness of these non- 
written forms of federal guidance on assessment, more than one-third (38%) of the 37 
states responding rated it as poor, while 30% rated it fair, 30% rated it good, and 3% 
rated it excellent. These ratings for timeliness are shown in Figure 2-E. 

The major concern expressed during our interviews was that the federal govern- 
ment, at that time, had not yet released final Title I regulations, and the draft regulations 
were generally perceived by states as too vague to help them move forward with con- 
fidence that they were following the rules of the law. Furthermore, states were deliber- 
ately holding back, since they believed that the policy might change and did not want 
to make adjustments that ultimately would not be necessary. Existing state accountabil- 
ity systems were put in place to serve state goals and purposes, not federal ones. 
Changing a state accountability system may involve amending state statutes or regula- 
tions, which can be a lengthy process, and this added to states’ frustration. 

At the same time, many state officials were sympathetic with federal employees who, 
they felt, were probably doing the best they could amid difficult circumstances, a too short 
transition period, and their own limited access to information. State interviewees also 
often noted that information and communication from USED seemed to be improving. 
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FIGURE 2-B STATE RATINGS OF THE QUALITY OF FEDERAL 
WRITTEN GUIDANCE ON ASSESSMENT 



Poor 10% 




Source: Center on Education Policy state interviews 



FIGURE 2-C STATE RATINGS OF THE TIMELINESS OF FEDERAL 
WRITTEN GUIDANCE ON ASSESSMENT 



Excellent 5% 




Number of states 
responding = 41 

Note: Percentages do not 
total 100% due to rounding. 



Source: Center on Education Policy state interviews 
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FIGURE 2-D STATE RATINGS OF THE QUALITY OF NON-WRITTEN FORMS 
OF FEDERAL GUIDANCE ON ASSESSMENT (SUCH AS ORAL 
COMMUNICATIONS, WEBSITES, AND REGIONAL MEETINGS) 



Excellent 5% 




Number of states 
responding = 37 



Source: Center on Education Policy state interviews 



FIGURE 2-E STATE RATINGS OF THE TIMELINESS OF 

NON-WRITTEN FEDERAL GUIDANCE ON ASSESSMENT 



Excellent 3% 




Number of states 
responding - 37 

NOTE: Percentages do not to- 
tal 100% due to rounding 



Source; Center on Education Policy state interviews 
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State Actions 

The Center on Education Policy interviewed state officials in September, October, and 
November 2002, just a few months into the implementation of the new law. Many 
state officials expressed their approval of the continued emphasis on standards and 
assessments in NCLB, though a few expressed concern about the mandatory extension 
of standards-based assessments to every grade spanning 3-8. Several states said that 
NCLB is in many ways just an extension of the reform that was already taking place in 
their states. One indication of this is that putting new assessments into place as required 
by NCLB was not a major concern of states. Most states already had standards in place 
or were far along in the development of them. States had also relatively long-term rela- 
tionships with test developers and had begun planning for new assessments or augmen- 
tations for old assessments. Many of our interviewees expressed particular support for 
the idea of monitoring and focusing on subgroup performance, toward the goal of clos- 
ing achievement gaps. 

Several officials mentioned moving to one accountability system for all schools 
(rather than just Title I schools) as a positive change. First, one accountability system is 
easier from an implementation and management perspective. Second, as one official 
said, a unified cause encourages people to take a reform more seriously because it is for 
all students and schools rather than just a subset. 

At the time of our interviews, the main implementation dilemma facing states was 
how to develop their plans for accountability and adequate yearly progress in the 
absence of final regulations on these matters. Interviewees described the competing 
pressures they were working under: on the one hand, wanting to move quickly to for- 
mulate an AYP plan to submit to the federal government by the January 2003 deadline; 
on the other hand, holding back from making decisions or changing their current 
accountability systems until they had more detailed information about how to imple- 
ment the law correctly and how much flexibility would be allowed. It seemed unfair 
to the states that they were being held to the deadlines even though the federal gov- 
ernment had not stuck to the agreed-upon timetable for its own actions. 

These concerns were corroborated by a November 10th statement from the 
Council of Chief State School Officers (CCSSO, 2002a), which noted that states had ° 0 o 0 ° 
received mixed messages about what may or may not be permissible from U.S. 29 

Department of Education personnel and which asked the Department to reconsider 
the deadline for state plans, in order to give states time to thoughtfully develop their 
plans. 

The question of how much flexibility would be permitted was central. States were 
hoping that USED would grant them some flexibility in implementing the law so they 
would not have to completely revamp their existing accountability systems or identify 
large proportions of their schools as failing. They were also awaiting further informa- 
tion about such issues as whether they would be allowed to set different starting points 
for different subgroups. As described in Chapter 1 of this report, states saw encourag- 
ing signs in Secretary Paige’s letter to the states of July 24, 2002 and in the draft regu- 
lations, both of which adhered closely to the language in the law but said the intent was 
not to dismantle systems that states already had in place. 
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The final regulations, as well as statements from Department officials, indicated that 
USED will be exacting in its interpretations of the law’s accountability provisions. For 
example, the final regulations made clear that states must use a single starting point for 
all subgroups. At the same time, however, the preamble to the final regulations said that 
a state may continue to use its current accountability system “if that system integrates 
adequate yearly progress as defined in the statute and regulations into its system.” The 
regulations offer no explanation of how this integration could or should be accom- 
plished, so the meaning of this statement may only become evident as the Department 
approves or disapproves state accountability plans. 

The Department may have been grappling with a conundrum posed by the aggres- 
sive nature of the improvement envisioned in the Act. The Administration has made the 
c O° c > 0 accountability provisions a cornerstone of its approach to education reform and is now 

faced with the problem that those provisions (as written by Congress), strictly enforced, 
could mean that large numbers of schools will be identified as needing improvement. 
If the Department enforces these provisions very strictly, several states with workable 
accountability systems in place would have to greatly modify or perhaps abandon their 
systems, instituting a new system that complies more directly with the provisions of 
NCLB. One possible solution would be for the Department to use its waiver authori- 
ty to allow those states to continue with their current systems or to give states flexibil- 
ity in certain portions of the accountability requirements, such as computation of the 
“starting point.” Another possible solution would be to grant provisional approval to 
states with accountability systems that appear to be working as long as the states con- 
tinue to show increased student achievement. However, these options would give the 
appearance of backing away from the portion of the Act that has been referred to as its 
cornerstone. 

Meanwhile, states were making progress on several fronts in their assessment and 
AYP plans. States were in the process of: 

0 Identifying schools on the improvement list for the 2002-03 school year based on 
the state’s criteria under the prior law. 

Oil Planning and negotiating with contractors for the development of content stan- 
dards and assessments for missing grades. 

El Identifying additional indicators for elementary and middle school levels. The great 
majority of states that had decided on their additional indicators by the time we 
interviewed them planned to use attendance data; a few states chose grade reten- 
tion data and assessments of other subjects, including science and writing. 

□ Considering different AYP models and, in some cases, running projections to find 
out how many schools will be identified as needing improvement under the vari- 
ous options. 

Putting in place annual testing at grades 3-8 presents a significant task for many 
states. However, our interviews revealed that while states are working hard to deal with 
this requirement, it is not the area of greatest concern. Virtually all of the state officials 
we spoke with indicated that the most formidable challenge is coming up with a sound 
plan for measuring AYP that will meet the numerous and complex rules in NCLB, 
while at the same time resulting in valid and beneficial decisions about schools. 
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As regards the special assessment requirements for English language learners, many 
states were initiating statewide efforts to test English language proficiency for the first 
time. It is too early to tell what issues states will face in developing these systems, but 
the process is likely to be a challenge for many states and school districts, especially 
those that have students from dozens of dominant language groups. 

All of the states we interviewed said that they plan to test the English proficiency 
level ofELLs in the 2002-03 school year. Among the most common assessments cho- 
sen by states were several commercially developed assessments of English proficiency, 
including the Language Assessment Scales (LAS), IDEA Proficiency Test (IPT), the 
Woodcock-Munoz assessment, and the Maculaitis Assessment for Competencies Test 
for English Language Proficiency (MAC II). At the time of our interviews, the U.S. 
Department of Education had not issued any ruling about whether these commercial 
assessments meet the requirements of NCLB (Testing English-language learners, 2002). 
NCLB also makes competitive grants available to consortia to develop better assess- 
ments of English proficiency that meet the law’s requirements. 



Issues to Watch 

Our interviews with states and our review of the research literature revealed several is- 
sues of concern associated with the assessment and accountability requirements of 
NCLB. Some of these are practical concerns, while others are technical concerns about 
assessment and accountability instruments and methods. 



Practical Concerns 

1. Large numbers of schools will be labeled as needing improvement, includ- 
ing many that are not failing. 

State officials predict that the NCLB requirements will result in a great number of 
schools being identified as needing improvement, many more than under the previous 
legislation, where numbers were already substantial for several states. The primary con- ° G G ° 
cern is that overidentification of schools will spread resources too thinly and subvert ° 

the law’s intent, which is to target additional money and help to the schools that need 31 

it most. One respondent likened the situation to a teacher having “too many students 
in one class... Some will slip through the cracks.” Another state said that with so many 
schools potentially identified “more time will be spent on compliance than on im- 
provement.” However, it could be argued that the intent of the law is to identify all un- 
der-performing schools, no matter how numerous. 

Another frequent concern was that NCLB provisions for identifying schools in 
need of improvement could damage schools and weaken public education in general. 

Several respondents said that the language in NCLB concerning failing schools and 
persistently dangerous schools would place an “embarrassing label” on schools that they 
would find difficult to ever “shake.” Some respondents were concerned that these labels 
would oversimplify what was taking place in schools and fail to acknowledge what is 
going right in these schools. A few state officials said they thought that NCLB had 
intentionally set very high performance requirements to undermine public education. 
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They said that if many schools failed to meet these benchmarks, then some policy- 
makers could make the case that public education is failing and advocate private school 
tuition vouchers as the solution. One respondent called NCLB “a backdoor to vouch- 
ers,” although the Department has repeatedly denied this charge. 

Overidentification of schools and districts in need of improvement is virtually 
inevitable because if even a single subgroup fails to make adequate yearly progress on a 
single test, the whole school is labeled. Some researchers have conducted analyses to 
compute the number of schools that would have been identified as in need of improve- 
ment if NCLB criteria were applied to past years’ test results. Most approaches yield 

large numbers of schools. 

32 & 

o o o □ A study by the Congressional Research Service (Smole & Riddle, 2001) estimated 

that 17% to 64% of the schools serving grades 3-8 in three states — Maryland, 
North Carolina, and Texas — would have failed to meet the AYP requirement for 
two consecutive years, if the NCLB Act had been in effect during school years 
1997-98 through 1999-2000. The highest estimates — up to 64% of schools — were 
for North Carolina, the only state that had data available for every student subgroup 
to be broken out under the new law. 

□ The Council of Chief State School Officers (2002b) ran AYP simulations using 
data from eight states. The analysis revealed that the percentage of schools that 
would be identified for improvement after two years ranged from 49% to 88%. 

Some say that the results of such studies are not entirely predictive, however, 
because the test data are from a time when the law was not in place and educators prob- 
ably were not paying as much attention to raising the test scores of each subgroup as 
they now will under NCLB. 

Although virtually all of our interviewees said that they thought the new law 
would certainly increase the number of schools identified as in need of improvement 
in their states, one official expressed the more positive expectation that over time, the 
new system will actually decrease the number of schools identified, explaining that, 
“One of the intended effects of public promulgation of data and corrective action is 
that there will be more attention paid to improving student achievement.” 




2. States are not starting on a level playing field. 

The rigor of states’ content standards, tests, and performance standards vary greatly; 
consequently, the percentage of students who score at the proficient level or higher 
varies accordingly. This means that if current tests and standards are used to set AYP 
objectives, some states will have much farther to go and will have to set much more 
demanding objectives than others, not necessarily because their students are achieving 
less but because their definitions of proficient performance are more stringent (Linn, 
Baker & Betebenner, 2002). 

For instance, Linn and colleagues found that Mississippi reported that 39% of 8th 
grade students scored at the proficient level or higher in mathematics in 2001, while only 
7% did so in Louisiana. The percentage of students who “passed” the 8th grade math test 
in Texas that same year was 92%. Although there may be real differences in math achieve- 
ment in these three states, the differences surely are not as great as the disparities in these 
percentages. On the National Assessment of Educational Progress (NAEP), which is given 




to a sample of students in each participating state and therefore can be used to make state- 
by-state comparisons, only 25% of Texas 8th graders scored at or above the proficient 
level, compared with 12% of Louisiana students and 8% of Mississippi students. So while 
Mississippi claims that four times as many of its students are proficient on its state test as 
Louisiana claims, the NAEP results rate more Louisiana students proficient. Obviously, 
Louisiana and Mississippi are not using the same definition of proficiency. 

An analysis by Education Week (Olson, 2002, February 20) explored more broadly 
what “proficient” means across all of the states, by using NAEP as the benchmark for 
comparison. Using data from the 2000 school year, this study compared the percentage 
of students who scored at or above the proficient level on a states own assessment with 
the performance of students on NAEP. The relationship between NAEP and state 
assessment results varies widely by state. 

M Most states had a greater proportion of students scoring proficient on their state 
tests than on NAEP, suggesting that those states’ proficiency levels are less rigorous 
than NAEP’s. For instance, in Oklahoma, 71% of 8th graders scored proficient on* 
the state math test, but only 19% of them scored proficient on NAEP In Virginia, 
61% of 8th graders scored proficient on the state math test, compared with 26% on 
NAEP 

H In a few states (such as Wyoming, South Carolina, and Kentucky), the proportions 
of students scoring proficient on both assessments were similar. 

il Only a handful of states (Idaho, Louisiana, Rhode Island) had a smaller share of stu- 
dents scoring at the proficient level on their own tests than on NAEP in 4th or 8th 
grade math, suggesting that those states’ tests may be more rigorous than NAEP 

Clearly, “proficiency” takes on very different meanings across the states. This is not 
a new problem; it was also an issue under the previous law. But the fact that states are 
not starting on a level playing field has greater significance now that the consequences 
attached to demonstrating AYP are more serious. 



3. There are incentives to lower performance standards. 

States had set the performance standards (often called achievement levels) for their state 
tests before NCLB declared they would be used to determine AYP objectives. Most states 
set performance standards under the more permissive requirements of the 1994 law, with 
“partially proficient” or “basic” achievement being the target. Some states set the mark 
for proficient at a level that yields results similar to the State NAEP results for their states. 

In some cases the proficiency levels are set so high that it may be unrealistic to 
expect all students to reach that level by 2014. No state is close to having all students 
meet the high standards set for proficient performance on NAEP or similar standards 
on many state assessments (Linn, 2000). Nationally, just 26% of 4th and 27% of 8th 
graders scored proficient or higher on the NAEP mathematics test. The generic defi- 
nition for the NAEP proficient level is mastery of challenging subject matter for the 
grade level (National Research Council, 1999). 

As a result, NCLB may unintentionally encourage states to lower academic standards 
since it continues to allow each state to define proficiency as it wishes, placing more 
emphasis on the goal of 100% proficiency by 2014 than on the rigor of the standards. 
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States with higher standards for their students and tougher definitions of proficiency will 
inevitably identify more schools as in need of improvement than states with lower stan- 
dards and so appear to be farther behind in the goal of having all students reach profi- 
ciency. Thus, states are punished for having higher standards. The fear that almost all of 
their schools will be identified as failing may push some states with relatively rigorous 
assessments to water down their test questions and/or lower performance standards. 

One official that we interviewed said that the level labeled proficient on the 
Stanford-9 Achievement Tests, a nationally standardized test used to measure student 
achievement in that officials jurisdiction, was too high to be used for AYP purposes. 
Another state said it was contemplating changing the labels of the performance levels 
on its state assessment, so that the level now referred to as “satisfactory” (which is not 
0 0°C> 0 the highest level on the test) would become the proficient level for federal AYP pur- 
poses. A different state plans to collapse two of its current levels — proficient and par- 
tially proficient — into a single category of proficient, essentially lowering the standard 
for what is acceptable performance. 

Some other states said they are committed to leaving the performance levels on 
their tests as they are. These leaders felt their states put a great deal of effort into deter- 
mining their achievement levels, including garnering public support, and they worried 
about what kind of signal it would send to the citizens if they suddenly lowered aca- 
demic expectations. Furthermore, the NCLB Act requires states to periodically admin- 
ister NAEP assessments and compare the results to state test scores, in part as an exter- 
nal check on whether state performance levels are sufficiently rigorous, although with 
no consequence besides the publicity. 

The indications that some states were considering lowering their standards led 
Secretary of Education Rod Paige to charge in an October 22 letter that state actions to 
lower standards were “not worthy of a great country” and called those who advocated the 
action “defenders of the status quo” and “apologists for failure” (USED, 2002). However, 
the letter did not mention the unevenness of the states’ playing field, discussed above, and 
it ignored NCLB s incentives to lower standards. The Secretary also did not point out that 
some states had low standards at the outset, and that NCLB offers no incentive for them 
to raise their standards. The more appropriate response might be to modify the law to give 
states that have lower expectations for achievement a reason to raise them. How states 
respond to the competing pressures is a major issue to track in the coming months. 



4. Every school and subgroup within a state will be measured from the same 

starting point. 

Many state officials were urging the Department to allow different starting points for 
different schools and subgroups, putting them on varied trajectories for improvement 
but always nudging them toward the same goal of 100% proficiency in 12 years. How- 
ever, the final regulations ended up ruling against this, allowing different starting points 
for different subject areas and grade levels only. 

Since AYP will be measured from the starting point and since all schools and 
groups will have to progress from the common starting point to the first objective in 
the state’s AYP plan, schools and groups that are well below the common starting point 
will have to move very far to avoid identification. Conversely, schools and groups 
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already above the starting point will not have to move at all, or could even show a 
decline and still meet the AYP requirement. Table 2-A presents a simple illustration. 



5. Testing and accountability systems that have been working may need to be 

dismantled. 

Officials in some states are concerned that the requirements of the new law are so spe- 
cific that it may be difficult to continue with their existing testing and accountability 
models, even when those programs are working well and are the product of many years’ 
development. It is unclear to what degree this will be the case in states, but the prospect 
of spending the time necessary to develop and pilot a new accountability system and 
then trying to build support for it was daunting for these states. They feared that having 
to “start from scratch” could impede the progress and positive movement being made. 

Perhaps in an effort to direct limited testing resources to the requirements of 
NCLB, many states are scaling back on tests that they currently have in subject areas 
other than math and reading, or on costly open-ended test questions that have to be 
scored by hand (Hoff, 2002). Although the prior federal law similarly offered no incen- 
tive for states to test subjects other than reading and math, it demanded far less in terms 
of testing time, administrative effort, and testing costs. To be able to comply with the 
new testing mandates at a time when state budgets are being cut in many places, states 
appear to be curtailing non-mandated aspects of their assessment programs. For 
instance, Oregon plans to save money by eliminating its writing tests for grades 3, 5, 
and 8, its science assessments in grade 5 and 8, and the extended-response portions of 
its 5th and 8th grade math exams. Massachusetts will not use history or social studies 
tests this year. Maryland, long known for its innovative assessments that presented stu- 
dents with a variety of rich, open-ended tasks, is replacing that program with a pre- 
dominantly multiple choice test, in part because NCLB requires states to provide indi- 
vidual student scores, which their former tests may not have been designed to do. 

Many people concerned about the impact of tests on instruction would describe 
these as movements in a negative direction, since it is well known that what gets test- P ^ 
ed is what tends to get taught, especially under high-stakes testing situations. 
Eliminating the more complex assessment tasks and tests of important areas like writ- 
ing and social studies simply because they are not required by the law may uninten- 
tionally send the message that those academic areas are unimportant and lead schools 
to reduce the time and effort they spend on these topics. However, others would argue 
that by requiring assessment and accountability in the areas of reading and math ini- 
tially, NCLB will help ensure that at least students have mastered those very funda- 
mental competencies.There are already plans to phase in science, and other subject areas 
could be added in the future. 

Meanwhile, states are exploring a variety of options for filling in the gaps in their 
testing programs to meet the new NCLB testing requirements. Table 2-B summarizes 
options that states are pursuing, each of which has advantages and disadvantages. 

States are also struggling with how to integrate the new requirements for measur- 
ing AYP into their existing accountability systems, again, to avoid having to redesign 
those systems from scratch. For example, North Carolina’s accountability system has 
been operating for the past six years and is functioning well; state officials are hoping 
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TABLE 2-A VARIABILITY IN PROGRESS REQUIREMENTS FOR DIFFERENT 
SCHOOLS IF THE 2003-04 AYP TARGET IS 50% PROFICIENCY 


PERCENTAGE OF STUDENTS 
SCORING PROFICIENT OR ABOVE 
IN 2001-02 


NEEDED INCREASES IN 
PERCENTAGE SCORING PROFICIENT 
OR ABOVE BY 2003-04 


School A: 30% 


School A: 20% 


School B: 45% 


School B: 5% 


School C: 75% 


School C: could decline by up to 25% 


Source: L/nn, 2002 . 



TABLE 2-B STATE OPTIONS FOR FILLING IN GAPS IN TESTING PROGRAMS 


OPTION 


PROS 


CONS 


Develop new customized cri- 
terion-referenced test (CRT) 


Aligned to state standards 


Expensive, takes several 
years to develop 


Use off-the-shelf norm-refer- 
enced test (NRT) augmented 
with some state customized 
test questions 


Somewhat aligned with state 
standards yet less expensive 
than completely customized 
test 


Less well aligned to state 
standards than fully cus- 
tomized test 


Let districts use their own 
local tests 


Allows local control, relieves 
some of the state burden 


Lack of comparability across 
districts, requires more state 
monitoring to assure regula- 
tions are met 


Collaborate with other states 


Efficient, states can combine 
resources 


Difficult to get agreement 
across states on content 
and other characteristics 
of the tests 


Source: Center on Education Policy interviews with state officials 
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to keep as much of it in place as possible (see the case study at the end of this chapter). 
In the state s current system, schools get placed into six different categories, depending 
on overall (not disaggregated) test performance. The two highest levels are Schools of 
Excellence and Schools of Distinction. The concern is how, in the future, to explain to the 
public that a school in one of these categories may not have made AYP. It is bound to 
happen that one or two subgroups will not make AYP in some schools that do very • 
well overall. This presents a big communications challenge, yet North Carolina does not 
intend to completely reinvent its basic accountability system. Another significant chal- 
lenge may be explaining to the public that the Title I sanctions do not apply if the 
school is not receiving Title I funds. 

Some states, like Kentucky, rate schools based on an accountability index that com- 
bines student performance across a number of subjects, rather than setting separate tar- 
gets in math and reading as the new federal law requires. In addition, Kentucky’s and 
some other states’ indices “give credit” for any positive movement in test scores across 
the range of achievement levels (for instance, more students reaching the basic level and 
fewer students below basic, compared to the year before), not just increases in the num- 
ber of students reaching the proficient level. The final regulations do not explain how 
indices such as those described above can be “integrated” with the highly specified def- 
inition of AYP in the new law. 



6. Twelve-year targets must be set but tests are changing. 

The results from state tests administered in the spring of 2002 form the baseline for 
defining AYP But during the 12-year period covered by the law, most states will be 
changing their tests or adding the new tests for additional grades required by the law, 
which will render some of the baseline information useless. States will probably have 
to set AYP targets now for the tests they have in place, and re-establish their targets 
when new tests come online in the next few years. In essence, they may have to start 
again from a new baseline in measuring AYR 

For some states that are introducing new testing programs in the 2002-03 school 
year, this presents an immediate dilemma. For instance, Texas and Maryland will not 
have statewide results for their new tests until spring 2003, after the new tests are first 
administered. They will not be able to measure change until the second (2004) test 
administration. 

States currently tend to revise and update their assessments every several years to 
stay aligned with trends in curriculum and instruction. However, NCLB seems to dis- 
courage states from updating and making their assessments more rigorous over the next 
12 years, because such changes would require resetting their AYP targets and interme- 
diate steps, and would likely cause a “set back” in meeting their AYP goals. The law also 
discourages states from gradually raising performance standards on their existing tests 
during this time period, preventing states from gradually raising academic expectations. 
The final regulations are unclear about how changes in assessment systems will impact 
baselines and AYP calculations over the course of the 12-year timeline for ensuring that 
all students are proficient, but, in the appendix containing comments, the Department 
promises that examples of ways in which states can move to new assessments will be 
included in nonregulatory guidance in the future. 




49 



Center on Education Policy 



From the Capital to the Classroom 



7. The law allows states a short time to turn around and report test data. 

The requirement for quick turnaround of results threatens the accuracy of the data that 
will be used for accountability. NCLB requires states to identify schools in need of im- 
provement before the start of the school year. Most states test in the spring, return re- 
sults to districts sometime in the summer or early fall, ask the districts to carefully re- 
view their data and report ariy potential errors to the state, and then check and correct 
all errors at the state level before releasing the final results to the schools and the public 
sometime later in the fall. Given the short turnaround time allowed for reporting tests 
results under NCLB, some states we interviewed were worried that they will have to 
sacrifice some of their quality control procedures, or have districts identify schools in 
need of improvement before the verified state results are released, which inevitably will 
q O O q lead to some schools initially being misidentified. 

A related problem is that with more testing per state and states testing at similar 
times, the competition for vendors’ (test publishers’) time and space may result in high- 
er costs, and greater volume may lead to more vendor errors. Testing organizations have 
already had to admit errors in scoring and reporting of state test results that have in 
some cases caused students to be wrongly held back or even denied high school diplo- 
mas. Many educators worry that a tight timeline for grading, less quality control, and 
the increased testing attributable to NCLB will result in many more errors. 



8. There are costs beyond the obvious ones of developing more tests. 

The law states that the schedule for administration of the additional assessments is con- 
tingent upon Congressional appropriations of at least $370 million for FY 2002 and 
$380 million in FY 2003 for states to help with the costs of developing and imple- 
menting annual assessments. Congress has actually appropriated $387 million for FY 
2002, and the President s budget for FY 2003 calls for the same amount, in both cases 
exceeding the minimum amount required. 

The majority of the officials we interviewed thought their state testing budgets, 
along with the federal funds reserved for this purpose, should be sufficient for their state 
to initially develop the required tests at additional grades. Some states still expressed 
concerns, however, about how to afford the long-term costs of maintaining, adminis- 
tering, and updating tests and analyzing test data. One state reported that as long as fed- 
eral funds are available for test administration (in addition to development), the state 
would meet the requirements. However, officials in this state did not expect the state leg- 
islature to step in and pick up the tab in the absence of federal funds for administration. 

There are additional, less obvious costs implied by the new law that federal leaders 
seem not to have taken into account. For instance, most states are already set up to disag- 
gregate test data by subgroups, but many are not prepared to disaggregate and report on 
other indicators such as graduation and attendance rates. Gearing up for these additional 
data collections and analyses will require resources. There are other costs related to imple- 
menting the new reporting requirements, strengthening testing expertise and capacity 
among local district administrators, and integrating the student test data with the teacher 
qualifications data to determine which teachers are teaching which students and in which 
subject areas. These are all coming at a time when state economies are declining. 
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Technical Issues 



1. School test results are volatile. 

Under the No Child Left Behind Act, important decisions about which schools are 
cited for improvement and which are subject to interventions, sanctions, or rewards 
will be based on test data that are inexact and changeable, even when the tests are well- 
designed and properly administered. School test results often fluctuate from year to 
year for many reasons that have nothing to do with how much students are learning or 
the quality of instruction. For instance, changes in the population of students being 
tested can produce significant year-to-year variations in average test scores. Countless 
factors can change the composition of the student body from year to year, including an 
influx of immigrants or refugees who are English language learners, an exodus of stu- 
dents to charter schools, or the construction of an upscale housing development in the 
neighborhood. 

There are other factors that contribute to what testing experts refer to as a test s stan- 
dard error of measurement. A students test score on any given day could be affected by 
the particular sample of questions included on that version of the test out of all possible 
questions that could be asked about the subject; the students physical condition or state 
of mind; distractions outside the classroom like construction noise or a barking dog; and 
factors such as lucky guesses on the students part or errors in scoring. 

Kane and Staiger (2001) estimate that more than 70% of the year-to-year fluctua- 
tions in average test scores for a given school or grade can be attributed to variations 
in student population and other external factors, rather than to changes in learning. And 
calculations of progress that are based on the difference between one years score and 
the next years score tend to be even less reliable than either year’s score, because the 
difference is subject to measurement error in both the baseline and follow-up years. 

Box 2-A describes how the standard error for a particular test affected decisions about 
school awards in California. 

Studies show that a school that makes large test score gains in one year is more like- 
ly than not to do worse the next year. In one such study, Figlio (2002) examined 
school-level test score data from two Florida school districts in two consecutive win- o q q o 
dows of time. In three-fourths of the schools, the proportion of students who met the ° 

state proficiency standards improved in at least one of the two time windows. But in 39 

69% of these schools that showed improvement, a school that improved in one instance 
fell back in another. The researcher attributed this finding to measurement problems: 

Schools facing an unusually “bad draw” one year (such as a large number of students 
who, for some idiosyncratic reason, did very poorly that year) tended to bounce back 
in the next, while schools facing an unusually “good draw” one year tended to revert 
to more average performance the next year. Small sample sizes (groups of students being 
tested) are especially prone to this kind of volatility. 

The bottom line is that test scores need to be thought of as estimates rather than per- 
fect measurements. A rise in average tests scores one year does not necessarily mean that a 
school or subgroup is succeeding, just as a drop in scores does not always mean it is fail- 
ing. The natural volatility of average scores (especially for small groups) means that it may 
be unrealistic to expect schools to post the consistent test score gains demanded by NCLB. 
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BOX 2-A LESSONS TO BE LEARNED FROM CALIFORNIA'S SCHOOL 
AWARD SYSTEM 



California recently had an accountability system that focused on cash awards 
to schools, whereas the No Child Left Behind Act focuses on sanctions. Yet 
the two programs have much in common, and problems that arose in Califor- 
nia offer some hints of what may come when the new federal law gets fully 
implemented. 

Called the Academic Performance Index (API), California's accountability sys- 
tem was passed by the state legislature in 1999 and tied cash awards to 
school scores on the state test. However, a series of investigative reports in 
the Orange County Register in August 2002 (Sharon, Tapia & Campbell, 2002) 
revealed that there was a large margin of error associated with the test 
scores, averaging 20 points. This is a big margin in a system where a single 
point could make the difference of whether a school receives tens of thou- 
sands of much needed dollars. 

The state assigned an API score to every school based on the Stanford-9 test, 
an exam that compares students with their peers across the country. The state 
also gave each school a target score to meet out of a possible 1000 points. 
For instance, a school needing to score 650 to win money would miss out 
with a score of 649. But a 20-point margin of error meant that a 649 indicated 
only that the school's true scores is somewhere between 629 and 669, and 
that the system could not reliably pinpoint fine changes in school perform- 
ance. (The Register relied primarily on a comprehensive study by Richard Hill 
of the National Center for the Improvement of Educational Assessment; Cali- 
fornia officials agreed with his conclusions about the size of the margin of er- 
ror.) As a result, the Orange County Register concluded that a third of the 
$67.3 million the state planned to give to schools as rewards in the 2002-03 
school year would go to schools where the magnitude of improvement was 
not statistically significant and could be due to chance. 

Why was the API's margin of error so large? It was the result of several fac- 
tors. With any test, there is regular measurement error — a student's score 
could be influenced by distractions such as an illness, the weather, or a bark- 
ing dog. In addition, there was error associated with changes in the sample of 
students tested each year. Furthermore, in many California schools, the popu- 
lation of students changes so much from grade to grade that it is impossible 
to get a precise measurement of a cohort's progress from grade to grade. Fi- 
nally, there was the problem that a large proportion of students' scores are 
not counted because of API loopholes. 
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A particularly troubling finding was that predominantly white schools 
earned more of the cash rewards than diverse schools in the state. Like 
NCLB, the API system required a school to raise not only its overall API, but 
also the scores of each of its major ethnic and racial groups and poor chil- 
dren to win an award. But the experience in California shows that an unin- 
tended consequence of such a rule can be a system tilted toward schools 
with the fewest racial groups. About 58% of schools statewide with one 
major group won awards last year, compared with about 29% of schools 
with four or more groups, even though black and Latino students were im- 
proving more. Research by Kane and Staiger (2002) showed that in the first 
year of the API, the scores of diverse schools rose an average of 35 points, 
while the scores of predominantly white schools rose 33 points. Yet the di- 
verse schools received an average of three to six dollars less per student in 
awards than mostly white schools. The problem is that the diverse schools 
had more hurdles to jump — if any one group falls short, the entire school 
loses. NCLB will require even more subgroups to show increases in per- 
formance to avoid sanctions, so this phenomenon is likely to be even more 
pronounced. 

The API's margin of error was never openly disclosed by state leaders until 
the Register revealed the problem. This year, rewards are being suspended 
because of the budget deficit, though state officials have not ruled them 
out in the future. 

California's experience offers some lessons to be learned as federal and 
state leaders proceed with implementation of NCLB. Most importantly, it il- 
lustrates some of the pitfalls of basing high stakes decisions (whether it be 
awards or sanctions) on results from a single test, because no test is a per- 
fect measure. There will always be some error associated with test results, 
and leaders should consider this fact in their policymaking. Issues of meas- 
urement error should be made public, and the extent of error associated 
with states' tests should be monitored. Methods also need to be in place to 
ensure that all students are truly being counted in the final test results. 



Source; Center on Education Policy based on news articles in Orange County 
Register, August 2002 
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Averaging a schools or subgroups test score data over several years, as permitted 
under NCLB, could help. For example, states could compare the percentage of students 
in a school who scored at the proficient level in reading averaged over the 1998-99, 
1999-2000, and 2000-01 school years with the average over the 1999-2000, 2000-01, 
and 2001-02 school years. Figlio (2002) found that using such three-year rolling aver- 
ages reduced the number of unstable schools (those that improved one year and fell 
back the next, or vice-versa) from about 57% to 33% in the Florida districts he stud- 
ied. As implementation of the law proceeds, average test scores should be examined 
carefully to see what kinds of patterns emerge and what degree of changes might be 
attributable to measurement error or other kinds of test score instability. 

42 

Meanwhile, state officials are asking whether differentiated sanctions might actual- 
ly be more appropriate, in light of the real difference between schools that fail to meet 
AYP due to essentially random events, versus schools that demonstrate consistent pat- 
terns of failure with the total group or a specific student subgroup over several years. 
Under NCLB, the consequences for a school missing a single indicator are the same as 
for a school that fails to meet multiple indicators. 



2. Some schools and districts will not be held accountable for subgroup 

performance. 

For many schools and districts, particularly those in rural or demographically homoge- 
neous areas, there may only be a handful of students in some of the reporting subgroups. 
As discussed above, average test scores from year to year are likely to be unstable for small 
groups. The law takes this into account by excusing school districts from counting a sub- 
group in their progress determinations if the number of test-takers in that group is too 
small to yield statistically reliable information, or if the results would reveal personally 
identifiable information about an individual student. The appropriate minimum thresh- 
old is open to interpretation, since there is no magical sample size above which subgroups 
are likely to be “statistically reliable” (Kane & Staiger, 2002), although reliability does in- 
crease as the number of students goes up. 

At the time of our interviews, states were considering a wide range of numbers for 
the minimum subgroup size, from a low of 20 test-takers in a school or district to a 
high of 70. In some cases, states may be considering technical qualities of their tests 
when they determine the minimum subgroup size, but in other cases, it is likely to be 
more of a political or strategic decision on the part of state leaders. 

Although increasing the minimum subgroup size could help districts or states reach 
more valid conclusions about which schools have failed to meet achievement targets, it 
could also mean that large numbers of schools would not be held accountable for sub- 
group performance at all and that subpopulations of students with educational needs in 
those schools may be overlooked. One rural state reported that at a minimum group 
size of 30, well over half the schools in the state would not have enough students in any 
category to measure subgroup performance. An official from another rural state called 
it “unfair” that the subgroup rules would only apply to a handful of the larger districts 
in her state. 

NCLB does allow data to be combined across grade levels for a particular subject 
within a school or district. This will help to increase subgroup sizes once tests are in 
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place for all the grades 3-8. However, in states that currently test only once at the ele- 
mentary, middle, and high school levels, districts and schools will be making their ini- 
tial AYP determinations based on only one grade’s worth of test data, until their revised 
testing system is in place. 

Use of three-year rolling averages can also increase the total number of students in 
each subgroup. This is because rolling averages are computed by adding up the number 
of students in a subgroup over the three years, then calculating an average for the total. 
This has two consequences. First, the scores are likely to be more accurate because the 
instability factor described above has been partially reduced. Second, subgroups with 
numbers below the minimum in a single year may now have enough scores to meet 
the minimum number of test scores when averaged across three years; in other words, 
schools could report on more subgroups if they use three-year rolling averages. State 
decisions about the minimum size of subgroups and how to report averages should be 
monitored over the coming years. 

Some experts have suggested alternative methods for achieving reliability in AYP 
decisions while holding all schools — large and small — accountable for subgroup per- 
formance. For instance, states could use ‘‘confidence intervals,” a standard measurement 
technique that would allow them to consider schools to have made AYP even if they 
fell below their AYP targets, provided their scores were within the range of possible 
measurement error (CCSSO, 2002b). A drawback to this approach, however, is that such 
statistical techniques are relatively complex and difficult to communicate to the public. 



3. The more subgroups a school reports on, the more likely it is to be identi- 
fied as needing improvement. 

An unintentional outcome of the law is that more subgroups in a school present a 
greater number of hurdles to leap, so the likelihood of being labeled as needing im- 
provement is related to the number of groups, as well as to student performance. 

Whereas school districts with considerable racial, ethnic, and language diversity will 
be responsible for reporting on numerous subgroups of students, small or homogenous 
school districts will not be required to report on subgroups at all. In some sparsely pop- 
ulated, less diverse states, only a few districts may be required to disaggregate data by 
subgroups. Thus, the law is stacked against diverse schools with many different sub- 
groups of students (that are large enough to meet the minimum size criterion selected 
by the state), each of which must make adequate yearly progress. 

There are also questions that have not been answered in regulations or guidance 
about how to define major subgroups. Most states currently report the traditional five 
race/ethnicity categories (American Indian, Asian/Pacific Islander, Black, Hispanic, and 
White) with some adding a sixth category (Other). As of January 2003, federal agen- 
cies will report six subgroups based on the 2000 Census, but even this newer division 
leaves questions. For example, how should schools handle biracial or multiracial stu- 
dents? Similarly, should all English language learners be considered as a single subgroup, 
or should they be categorized by their specific language background, or should there 
be different subgroups for students who do not know any English versus those who can 
speak English but are still learning to read and write it? 
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4. English language learners and students with disabilities are different in 
nature than other subgroups. 

Two of the subgroups — English language learners (called limited English proficient 
students in the law) and students with disabilities — are different from the other sub- 
groups, because they are categorized more by students’ special, and in some cases tem- 
porary, learning challenges than on their background characteristics. 

While many state officials we interviewed liked the additional attention paid to 
ELLs and students with disabilities in NCLB, others had great concerns about includ- 
ing them in the AYP system and contended that these two groups should be treated 
44 differently for calculating AYP. 

0 0°0 0 Limited English proficiency, in particular, is defined in a way that makes measuring 

progress of students impossible from one standpoint, since English language learners 
who become proficient in English leave the group and are replaced by new ELLs, who 
by definition are not proficient in English. A few state officials interviewed said that it 
would be “impossible” to get all ELLs to the proficient level because the population is 
not stable — those who become proficient in English leave the English language learn- 
ing programs and new students are added. So a school that makes enough progress with 
its ELLs to mainstream them within a couple of years might actually look worse on 
tests comparing the ELL subgroup than a school whose ELLs advance only slightly. 

By including students with disabilities in testing, NCLB is trying to rectify schools’ 
past practices of keeping their scores high by excluding large numbers of students with dis- 
abilities from testing. At the same time, some students with disabilities have cognitive or 
other disabilities that will prevent them from ever performing at the proficient level. The 
law will allow school districts to exempt only a small proportion of their student enroll- 
ments from accountability testing for this reason — the Department originally suggested 
0.5%, but the final regulations call for the formation of a new group to determine what 
the acceptable exemption rate will be. Some state officials we surveyed were troubled by 
the proposed 0.5% limit, voicing concern that forcing children to take a test that is beyond 
their capabilities would damage them. One official said, “Treating everyone the same isn’t 
being equitable at all. I just don’t know if jumping through all the same hoops for all chil- 
dren is equitable.” It is not at all clear, however, what percentage of students with disabili- 
ties is appropriate to exempt; experience will shed more light on this issue. 



5. Making valid decisions about schools requires highly accurate data. 

States are struggling with some of the data collection requirements for NCLB. Most 
states are already set up to disaggregate their test results into most of the subgroups re- 
quired by NCLB, except several states are not yet prepared to do so for low-income 
students (those eligible for free and reduced-price lunches). In addition, states are not 
generally equipped to report on disaggregated attendance and graduation rate data. 
Thus, in some states, substantial resources will need to be devoted to expanding their 
data collection systems. 

Another set of concerns relates to the accuracy of the background information about 
students used to disaggregate results by subgroups. As one interviewee observed, “disag- 
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gregated tests results are only as good as the data that people supply.” Some states will 
probably leave it to test contractors or school districts to collect student background data, 
which in most cases means that the information will be filled in by the students at test- 
ing time. But research has shown that students frequently misreport this kind of infor- 
mation. For instance, one state realized that self-reported data from students about their 
free and reduced price lunch status were unusable because so many students left the ques- 
tion blank. In another state, 20% of students in matched records changed their ethnicity, 
and in a third state, 4% changed their gender. Some students report that they are “Native 
American” because they are born in the U.S. (Dougherty, 2002). Then there are multira- 
cial children, who may not be sure how to categorize themselves. 

Many states will need to devote significant resources to handling these new 
requirements. Some interviewees saw this as a positive consequence of NCLB — the law 
is spurring their states to develop more comprehensive and accurate data collection sys- 
tems, long due but not a high priority until now. 



Conclusion 

The assessment and accountability provisions are the heart of the No Child Left Be- 
hind Act. They are the main tools that the federal government will use to press states, 
school districts, and schools to boost the academic performance of all groups of stu- 
dents and to close racial, ethnic, and other achievement gaps. These are ambitious goals 
that states appear to be taking very seriously, based on what we learned from our nu- 
merous and often quite candid discussions with state officials. 

The assessment provisions of NCLB are complex and have attracted considerable 
attention in the media and public discussions. From what we have observed, states are 
making headway in implementing these provisions, perhaps because states have been 
working for several years with standards and assessments on their own and in response 
to the IASA requirements. The key here will be to reach a delicate balance between 
complying with the law and maintaining the features of existing state assessment sys- 
tems that are working well. °o° 

The most contentious issues that confront states are related to adequate yearly 45 

progress. Because of practical problems and technical factors, it could be difficult for 
states to develop plans for measuring AYP that will accurately target the schools that 
have not performed well without also incorrectly targeting schools due to measure- 
ment anomalies. States have legitimate concerns about how this system will play out, 
and the federal government’s delay in issuing final regulations and strict interpretations 
of the law did not help with their anxiety. 

The federal government, the states, and outside researchers should closely monitor 
how the accountability provisions of NCLB unfold in the coming years. Lowering stan- 
dards will not help the children who are receiving an inadequate education, but nei- 
ther will approaches so rigid that the goals of the law are subverted. 
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CASE STUDY 

WHAT NORTH CAROLINA MUST DO TO BRING ITS ASSESSMENT AND 
ACCOUNTABILITY SYSTEM IN LINE WITH THE REQUIREMENTS OF THE NO 
CHILD LEFT BEHIND ACT 



North Carolina's assessment and accountability system meets some of the 
requirements of NCLB, such as disaggregating test scores by student sub- 
groups. But it will be complicated, and perhaps not feasible, to reconcile other 
46 aspects of the state's system with NCLB, such as its "value-added" approach 

Q that looks at average growth in achievement of the same students from one year 

o° °o to the next. 

Like many other states, North Carolina had a statewide system of assessments and 
school level accountability in place before the No Child Left Behind Act was enacted. 
And like the systems in many states, the North Carolina system does not align perfect- 
ly with the provisions in NCLB. Following is a discussion of how the two systems com- 
pare with one another and what North Carolina plans to do to bring its system in line 
with the Act’s requirements. 

The North Carolina Approach 

ASSESSMENT 

North Carolina assesses all children statewide in reading and mathematics in grades 3 
through 8 and administers end of course tests in 10 subjects in high school. The state 
also administers a comprehensive test in reading and mathematics to students in grade 
10 in high schools that receive Title I funds. These assessments meet the current re- 
quirement for assessment in the Elementary and Secondary Education Act, but will 
need to be adjusted to meet the requirements that come into play under NCLB. As- 
sessment data are already disaggregated into the categories required by NCLB. 

To meet these new requirements, North Carolina will have to administer the high 
school assessments in grade 10 reading and mathematics and, by school year 2007-08, 
institute a science assessment at least once in each of three grade spans: grades 3-5, 6- 
9, and 10-12. 

The state is concerned, however, about the timetables in the law for reporting 
assessment results. Unlike some states that contract with outside vendors or groups to 
compile assessment results, North Carolina provides local school systems with hardware 
and software to scan and score answer sheets. The local systems then forward their data 
to the state department for additional quality control and statewide reporting purpos- 
es. In the past, if there were no changes to the assessment, the state could get official 
results to districts and schools in August. When changes occurred in the assessments, 
official school, district, and state results from the State Department of Public Instruction 
would not be provided until September or October. The state is concerned that the 
schedule in NCLB will leave no time for adequate quality control over results. 

ACCOUNTABILITY 

While the process of aligning North Carolina’s state assessment system to NCLB is 
rather straightforward, the situation with accountability is much more complex. The 
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North Carolina system for assessment and accountability was put in place for elemen- 
tary and middle schools in school year 1996-97 and in high schools the following year. 

The system contains incentives and sanctions and is based on results on the state assess- 
ments described above. 

The accountability system has several major features. 

□ For each school, growth expectations of performance were established using tests 
administered during the 1992-93 and 1993-94 school years, except for grade 3, 
where pretest scores collected in school year 1996-97 were initially used. The 
grade 3 formula has since been revised based on data collected in 2000-01. 
Expected growth rates differ from school to school. 

□ Results for each school year are calculated using matched student scores, to deter- 
mine the extent of growth students experienced from one year to the next. 

□ The assessment system has four levels of performance, with level III designated as 
proficient and level IV as advanced. 

□ Based on the overall percentage of students scoring at or above the proficient level, 
as well as a comparison of actual growth with expected growth rates, schools are 
divided into six categories, ranging from Schools of Excellence, the top category 
to Low-performing Schools, the lowest category. (Table 2-C shows numbers of 
schools in each category.) 



TABLE 2-C NUMBER OF NORTH CAROLINA SCHOOLS ATTAINING VARIOUS 
CATEGORIES ON THE STATE'S ACCOUNTABILITY SYSTEM 



RESULTS FOR THE 2001-02 SCHOOL YEAR ARE: 



CATEGORY 


NUMBER OF SCHOOLS 


PERCENT OF TOTAL 


Schools of Excellence 


299 


13.6 


Schools of Distinction 


640 


29.6 


Schools of Progress 


521 


23.8 


No Recognition Schools 


465 


21.2 


Priority Schools 


132 


6.0 


Low Performing Schools 


18 


.8 



Prior to NCLB, North Carolina identified Title I schools in need of improvement as those 
schools in the low-performing category as well as those schools with less than 60% of 
student scores below achievement level III. For 2001-02, 17 Title I schools were desig- 
nated as in need of improvement. 



Source: The ABCs Accountability Model: Results 2007-02. State Board of Education/ 
Department of Public Instruction, www.ncpublicschools.org 
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M Financial rewards are distributed to certified staff and teaching assistants in schools 
that attain or exceed expected growth. 

m The state provides technical assistance to low-performing schools. Local superin- 
tendents and, in some instances, the state board, may take further action. 

£9 While North Carolina has disaggregated data on student performance for each 
school and makes this information available to the public, it is the overall perform- 
ance of students, not the performance of subgroups, that determines a school’s des- 
ignation. 



Major Differences Between the 
North Carolina System and NCLB 

The North Carolina assessment and accountability system differs in several respects 
from the requirements of the No Child Left Behind Act: 

(S The North Carolina system is based on changes in the average scale scores of the 
same students from one year to the next, calculated by comparing the average scale 
scores for the group of students one year with average scale scores of the same stu- 
dents the next year. NCLB, by contrast, compares the percentage of students in a 
school, as a whole and by subgroup, scoring at the proficient level against a stan- 
dard set by the state following procedures established in NCLB. Therefore, the same 
children are not followed over time by NCLB. NCLB requires increased perform- 
ance of each subgroup in each school against the state standards whereas the North 
Carolina approach looks at overall changes in the average scale scores in the school 
as a whole. 

IS NCLB sets a time certain by which all children must attain proficiency, a concept 
not included in the North Carolina approach. 

9 North Carolina sets expected growth rates for individual schools, based on their 
individual performance in relation to overall state results. NCLB requires states to 
establish a single starting point and incremental increases that apply to all schools 
and subgroups; the performance of schools and subgroups is based on progress from 
the starting point and incremental increases. 

H NCLB requires that at least 95 percent of all students and of each subgroup par- 
ticipate in the assessments. North Carolina has a provision for the school as whole. 

In an effort to determine how the NCLB could affect the designation of schools 
into performance categories in North Carolina, the state conducted several simulations 
that applied NCLB requirements to actual test data from past years. For these simula- 
tions, the state used the following assumptions: the starting point was set based on 1997- 
98 data; the number of students assessed in a subgroup had to be at least 30 for the 
group to be counted for school designation; and the 10% reduction in numbers of stu- 
dents scoring below proficiency (the so-called “safe harbor” provision) was applied. The 
Act allows the safe harbor provision to be used only if the school made adequate 
progress on the additional indicator. However, the state did not have information on 
the additional indicator and, for purposes of this study, assumed that all schools had 
made required progress on it. 
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The state did six simulations, using different assumptions regarding periods for 
incremental increases and the use of a single starting point for all schools and groups (as 
final regulations now require) and use of different starting points for different sub- 
groups. Using these simulations, the state estimated that between 24% and 62% of its 
schools would make adequate yearly progress each year; that between 25% and 65% of 
those not making adequate yearly progress for one year would fail to do so for the sub- 
sequent year; that between 20% and 53% would fail the third year; and that between 
17% and 42% would fail the fourth year. 

Since these simulations were run, the US. Department of Education has issued final 
regulations for school accountability. Absent waivers, these regulations require the start- 
ing point to be the same for all schools and groups. The simulation for the approach 
that North Carolina now seems most likely to use showed that approximately 30% of 
schools would make adequate yearly progress each year. 

The major reason why so few schools would make adequate progress appears to be 
the performance of subgroups, even in schools that have overall high performance. 
While supportive of the requirement in NCLB that performance of each group should 
be included in measuring the performance of a school, the North Carolina Department 
of Public Instruction is concerned about adequately communicating to the public how 
schools identified under the state system as Schools of Distinction or Schools of 
Excellence could also be identified, under NCLB, as schools in need of improvement. 

The state is also concerned about possibly having to provide technical assistance to 
a large number of schools. 

Incorporation of NCLB into the North Carolina System 

For school year 2002-03, North Carolina plans to treat the AYP requirements in NCLB 
as an additional component of the state system. Schools will continue to be placed in 
the categories described above but will also be designated as having or not having made 
adequate yearly progress under the requirements of NCLB. After results for the year 
have been compiled, the state will decide whether any changes to the system should be 
made. 

One major change has already been made in the area of financial rewards for cer- 
tified staff and teaching assistants in schools that perform well. In prior years of the 
accountability program the certified staff and assistants received incentive awards for 
making expected growth and higher incentive awards for making high growth. Starting 
with school year 2002-03, the State Board of Education approved reducing the 
amounts of incentive awards for making expected or high growth but adding an incen- 
tive award for staff in schools that make AYP under NCLB. 



Source; Center on Education Policy state interviews 
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Providing Public School 
Choice and Supplemental 
Education Services 



KEY FUNDINGS 

□ The No Child Left Behind Act had very ambitious deadlines for implementing 
new requirements for public school choice and supplemental education services, 
and it was difficult for states and school districts to act on these provisions in fall 
2002 without data on which schools and students it would affect. The timing of 
the U.S. Department of Education in releasing final regulations and guidance 
also seems to have been a factor as to why states were not ready to fully imple- 
ment these provisions. 

□ In fall 2002, several states estimated that few parents had made use of the pub- 
lic school choice option thus far. This could be due to state and local delays in 
identifying low-performing schools, inadequate efforts to inform parents that 
the option was available, or the commitment of parents to their neighborhood 
schools. 

□ Practical constraints could make it difficult to implement the Act's choice 

requirements in districts that have overcrowded schools, have a single school at 
certain grade spans, or are located in rural or remote areas. In the first few 
months of implementation, some districts were limiting students' choices due to 
a lack of space in receiving schools, but the final regulations released in late °o° 

November said that lack of capacity cannot be used to deny choice for students. 53 

□ As of November 2002, only 15 states had developed an approved list of sup- 
plemental education service providers. Non-regulatory guidance issued by the 
U.S. Department of Education required states to have an approved list in fall 
2002. Many states were waiting for final regulations, while others did not have 
any schools that were required to offer these services this. school year. The num- 
ber of providers on state-approved lists varied widely, from a handful to more 
than 100. 




64 



Center on Education Policy 



Introduction 



n 






he No Child Left Behind Act requires states and school districts to take a va- 
riety of steps aimed at improving student achievement. Two of the most sig- 
k- nificant steps pertain to requiring public school choice and providing supple- 
mental education services when schools that receive federal Title I funds are cited as 
“in need of improvement” under the law s accountability provisions. 



The laws choice provisions require schools that have failed to meet the state s def- 
inition of adequate yearly progress for two or more consecutive years to allow students 
in these schools the option of transferring to a better-performing public school in the 
same district that is not in need of improvement. The requirements for supplemental 
education services apply to schools that have failed to make adequate yearly progress 
for three or more consecutive years. School districts must arrange for students in these 
schools to receive tutoring and other supplemental education services from a state- 
approved service provider selected by the student’s parents. These providers can include 
public agencies or private nonprofit or for-profit entities. 

This chapter describes how the federal government, states, and school districts are 
implementing public school choice and supplemental education services. It also dis- 
cusses some issues that states are currently facing and issues to watch in the coming 
years. 



Public School Choice 

Background 

The term “school choice” can be used to describe various policies or programs that 
give parents the option of choosing educational settings for their children other than 
their neighborhood public schools. Charter schools, open enrollment or choice among 
public schools, tuition vouchers for students to attend private schools, and home °o° 

schooling are some common forms of educational choice. The No Child Left Behind 55 

Act clearly limits choice to the public school system, allowing students in designated 
schools to transfer to a better-performing public elementary, middle, or secondary 
school or a public charter school. 

Proponents of school choice argue that giving parents greater choice among 
schools will force schools that are performing below standard to improve their practices 
or risk losing funds as students leave to attend “better” schools. Public school choice in 
the No Child Left Behind Act is intended to prevent children from languishing in 
schools that have repeatedly been designated as in need of improvement. 

The choice requirements of NCLB are an outgrowth of earlier provisions for choice 
in the 1994 Improving Americas Schools Act (IASA) that allowed, but did not require, 
school districts to use funds to develop and implement public school choice programs 
for Title I children. (The 1994 legislation also introduced the concept of labeling schools 
as “in need of improvement”) 
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In the federal appropriations bills for fiscal years 2000 and 2001, Congress required 
districts to offer public school choice to students in Title I schools identified for 
improvement. The money was distributed through either a formula or competitive 
grant process, as determined by the state, and only districts that had at least one school 
labeled as in need of improvement were eligible to receive the funds. In guidance issued 
by the U.S. Department of Education for the FY 2000 appropriations bill, the 
Department acknowledged that the additional funds may not be sufficient to offer a 
transfer to all students attending Title I schools identified for improvement and stipu- 
lated that in these cases, the neediest students should have priority in exercising this 
option. 

Evidence suggests that the choice provisions of these appropriations bills were not 
0 0°0 0 widely implemented. Alexander Russo (2002) suggests the choice program was under- 
funded, poorly publicized to parents, limited to low-performing schools, administered 
by unenthusiastic school officials, and full of bureaucratic loopholes. But in fairness to 
school districts, the choice provisions contained in the appropriations bills allowed 
exceptions. In guidance on this legislation, USED acknowledged that class size limits, 
desegregation orders and other capacity issues could make implementation difficult in 
certain school districts. Although there were no published USED reports on the imple- 
mentation of these prior choice requirements, anecdotal evidence suggests that many 
districts had legitimate reasons that allowed them to avoid offering choice to students. 
For these reasons public school choice was not widely implemented under the FY 2000 
and FY 2001 appropriations bills. 

The No Child Left Behind Act takes the prior legislation several steps further by 
linking the public school choice option to the school accountability system, earmark- 
ing funds for transportation, and requiring all Title I schools in school improvement, 
corrective action, and restructuring status to offer public school choice. NCLB also has 
expanded and heightened the level of enforcement of public school choice in Title I 
schools. 

Although pro-voucher amendments were defeated in Congress, some educators 
remain suspicious that the Acts public school choice provisions could pave the way for 
vouchers. The Departments treatment of the choice provisions in the final regulations, 
which allows virtually no flexibility in denying students transfers, has stoked these con- 
cerns among critics, who fear that private school vouchers could become the next step 
in cases where a districts better-performing public schools do not have space for trans- 
fers from lower-performing schools. 



Choice Provisions of the No Child Left Behind Act 

As explained in Chapter 2, schools that do not make adequate yearly progress in raising 
test scores, as defined by the state, will be designated as in need of improvement and 
must undergo a series of improvement steps and corrective actions including public 
school choice. Specifically, the Act requires schools that have not made adequate yearly 
progress for two consecutive years or more to allow students the option of transferring 
to another public school or public charter school in the same district that has not been 
designated as in need of improvement, corrective action, or restructuring. The Act also 
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BOX 3-A TRANSPORTATION FUNDS 



The No Child Left Behind Act requires each school district to set aside an 
amount equal to 20% of its combined Title I allocations (basic, concentration, 
targeted assistance, and finance incentive grants) to pay for transportation 
costs associated with the public school choice requirements and for supple- 
mental education services. NCLB allows the funds to come from federal, 
state, local, or private sources, as long as the dollar amount equals the 20% 
figure. For instance, funds can come from the Title I program, the Safe and 
Drug Free Schools and Communities program, or the innovative programs 
under Title V. Of this set-aside, 5% is earmarked for transportation for public 
school choice and 5% for costs related to providing supplemental education 
services; the remaining 10% can be used for either of these purposes. 



Source; P.L. 107-110 , the No Child Left Behind Act 



requires school districts to set aside funds to pay for transportation costs associated with 
the choice requirements. (See Box 3-A, Transportation Funds.) If the district cannot af- 
ford to pay transportaion costs for all students who apply, the law stipulates that the 
lowest-achieving students should have priority. These public school choice require- 
ments of NCLB took effect at the start of school year 2002-03 and applied to schools 
that had been designated as in need of improvement under the terms of the 1994 law. 

School districts can, but are not required to, meet the choice requirement by forg- 
ing cooperative agreements with other districts that allow students to transfer to a 
school in a different participating district. Furthermore, states can require school districts 
to employ interdistrict transfers (between two or more districts) as part of the correc- 
tive actions imposed on schools that have not made adequate progress for several years. 

The law contains another choice-related provision, the “unsafe school” choice 
option, which takes effect in school year 2003-04. Under this option, districts must allow 
students to transfer to a safe public school if the public school they are attending has been 
labeled “persistently dangerous” or if a student becomes a victim of violent criminal 
offenses at the public school he or she attends. States must develop a policy and plan for 
identifying schools that are persistently dangerous, and must submit this policy and plan 
to the U.S. Department of Education for approval. As of October 31, 2002, all states had 
submitted a plan to USED but not all plans had been approved. Specific information on 
the content of the plans was not available in time for this report. 
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Department of Education Actions 

LIST OF SCHOOLS IN NEED OF IMPROVEMENT 

On July 1, 2002, U.S. Secretary of Education Roderick Paige released a list of Title I 
schools that had been identified by states as “in need of improvement,” based on recent 
state test data (USED, 2002b). This list included 8,652 schools, broken out by state, that 
had not made adequate yearly progress based on the state measures developed under 
the 1994 IASA, the predecessor law to NCLB. Schools were identified using data from 
the 1999-2000, 2000-01, or 2001-02 school years. Most of these schools had been in 
program improvement status for one or two years, under the provisions of the prior 
law. States were permitted to remove schools from the list that had made adequate year- 
ly progress for two years based on 2001-02 assessment results, but they did not have to 
q O O q add schools that were not on the list already, even if those schools did not make ade- 
quate yearly progress in 2001 -02. The number of schools identified for improvement 
may have decreased because of this allowance or, as some states conjecture, the Depart- 
ment of Education may have overidentified schools in its original figures. In any case, 
most states we interviewed expected the number of schools in need of improvement 
to go up next year, when states begin using their new definitions of adequate yearly 
progress, so the number of children eligible for choice is likely to rise, too. 

The schools currently identified for improvement were required to implement the 
public school choice requirements at the beginning of the 2002-03 school year even 
though USED had not yet issued final regulations and guidance on public school 
choice by the start of the school year. 



LETTER TO CHIEF STATE SCHOOL OFFICERS 

On June 14, 2002, Secretary Paige issued a letter to the chief school officers in all states 
that included guidance about school choice and supplemental education services 
(USED, 2002a). The letter made several important clarifications to the law, most of 
which were repeated in the final regulations, but a few of which were actually retract- 
ed in the regulations. For example, while the June 14 letter suggested that health and 
safety code requirements were an acceptable reason for a district to restrict choice, the 
final regulations said health and safety factors could not be used to deny transfers. The 
June 14 letter did clarify that NCLB does not prohibit a district from spending more 
on transportation costs than the 20% set aside required by law or from using state and 
local funds to cover costs related to public school choice. 



FINAL REGULATIONS 

The final regulations for Title I, issued by the U.S. Department of Education on Novem- 
ber 26, 2002, tend to reiterate the language of the law.The final regulations deny the flex- 
ibility that school districts were hoping for on implementing public school choice and 
allowing exceptions to implementation under special circumstances. Districts with un- 
usual circumstances that prevent them from implementing choice are encouraged in the 
regulations to enter into cooperative agreements with neighboring districts, hire more 
teachers, or add more classrooms. 
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The regulations made several important clarifications to the law: 

□ Extent of choice . The final rule defines choice to mean, at a minimum, the oppor- 
tunity for eligible students to choose between at least two better-performing 
schools. 

□ Capacity limits. The major clarification in the final regulations asserts that school 
districts cannot use space limitations in receiving schools, including class size lim- 
its and health and safety code requirements, as a reason to deny students transfers. 

□ Class size limits and health and safety concerns. If a school district has reached its 
“maximum instructional capacity” or has class size limits set by state policies, it must 
still offer choice and adhere to the state-mandated limits. The regulations also require 
that choice be implemented in a healthy and safe manner. If a school has reached its 
state-mandated class size limits or maximum capacity in all classes, as regulated by fire 
codes or other health and safety concerns, USED suggests that it hire additional 
teachers and add classes to accommodate students who wish to transfer to the school. 

a Desegregation orders. In cases where a school district has a court-ordered desegre- 
gation plan in effect, the final regulations clarify that it should continue to adhere 
to the desegregation plan as ordered by the court and make choice available. If the 
school district is unable to implement public school choice consistent with its 
desegregation plan, USED requires it to “secure appropriate changes” from the 
court. The Department leaves open how it will handle cases where the courts deny 
amendments to desegregation orders, but indicates that it does not intend to with- 
hold Title I funds from states that make good faith efforts to comply with the law. 

03 Cooperative agreements with neighboring districts. In school districts in which all 
schools have been identified for improvement, the final regulations indicate that 
this situation cannot be used to deny public school choice to eligible students. The 
regulations encourage but do not require districts to form cooperative agreements with 
neighboring districts in order to allow students to transfer. 

03 Use of supplemental education services, corrective action or restructuring. If the school 
district cannot provide public school choice under these options, the regulations 
allow districts to use any of the improvement methods outlined by the statute for 
later years in the process. For example, some districts in rural and remote areas have 
already indicated that they plan to provide supplemental education services in this 
situation. The regulations justify this by noting that these methods are already per- 
missible uses of federal funds under Title I. 

ffl Grounds for not implementing public school choice. Under the regulations, a student 
can be denied choice only in the very limited situation where state law specifical- 
ly prohibits certain kinds of choice. In particular, a school district “may invoke the 
state law prohibition on choice . . . only if the state law prohibits choice through 
restrictions on public school assignments or the transfer of students from one pub- 
lic school to another public school.” 
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On December 9, 2002, the U.S. Department of Education issued draft nonregula- 

tory guidance for the choice provisions. Key points made in the guidance include the 

following: 

□ States must annually report the number of students and schools that participate in 
public school choice. 

□ School districts are not required to cover transportation costs after the end of the 
year in which the school of origin is no longer in improvement status. 

□ School districts in rural and remote areas that have no practical choice options for 
students attending schools in improvement status may offer supplemental education 
services in lieu of public school choice. 

□ School districts that must seek amendments to desegregation orders from the 
courts in order to provide choice can use Title I money to pay for associated legal 
costs. 

□ Title I public school choice can become part of a state’s open enrollment program 
as long as the requirements ofTitle 1 are met, allowing children who are the low- 
est achieving and come from low-income families to exercise choice first. 

□ As with pre-NCLB legislation, Title 1 funds do not follow a child who transfers to 
a non-Title I school. 

□ School districts must still offer choice if transportation funds are not sufficient to 
pay for all students who request transfers, but the district is only required to spend 
up to the set-aside required in the legislation. 

□ School districts are not required to pay for transportation costs for students who 
left a school in improvement status prior to the enactment of NCLB. 

□ School districts may establish transportation zones within an LEA based on geo- 
graphic location of schools, as long as more than one choice is offered. 

□ Parents should be informed of their child’s eligibility through multiple methods, 
such as mail, newspapers, posters, and the Internet. 



State and Local Implementation 

While many of the state officials we interviewed spoke about school choice, they 
downplayed its importance. The reasons why were not clear, although some states not- 
ed that in many districts, there are no realistic options to which students will be able to 
transfer, either because potential schools are overcrowded or because there are simply 
no other schools. 

State and local implementation of the NCLB choice provisions was complicated 
by the very ambitious deadlines required by the law. Often results from spring testing 
do not become available until the fall, and when we interviewed states in the fall, some 
still had not finished analyzing their test score data or had not received test data from 
outside test contractors. Consequently, these states had not yet determined which 
schools were in need of improvement and which students would be eligible for choice. 
Furthermore, final regulations and guidance were not yet available at the time school 
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districts were supposed to begin implementing choice. Some states used the law and 
the Secretary s June 14 letter to develop draft guidance for their school districts, while 
other states directed local school people to the Department s NCLB website, where the 
letter is posted. 

Some states and school districts already have “open enrollment” policies, which 
allow students to transfer to schools within the same district (intradistrict) or in anoth- 
er district (interdistrict). According to the Education Commission of the States (2002), 

33 states have mandatory or voluntary intradistrict and/or interdistrict laws that allow 
students to exercise public school choice. In addition, 41 states have charter school laws, 
which permit students to enroll in charter schools that operate with less regulation and 
more flexibility than other public schools. These existing state and local choice policies 
will interact with the federal choice provisions in various ways. On one hand, the exis- 
tence of open enrollment or charter school policies can make it difficult to track which 
students are exercising choice stemming from NCLB and which are changing schools 
for other reasons. On the other hand, experience with existing public school choice pro- 
grams can help states and districts to implement the NCLB choice provisions. The St. 

Paul school district, for example, reports that it used its existing school choice system and 
magnet schools as a basis for implementing NCLB choice provisions (Welbes, 2002). 

Since districts are directly responsible for implementing the NCLB choice provi- 
sions, it is necessary to go to the district level to answer such key questions as what is 
being done to carry out the requirements and how many students have chosen to trans- 
fer thus far. Because implementation of choice is proceeding in various ways and at dif- 
ferent paces, a state-by-state account cannot yet be given. The more in-depth data col- 
lection and analysis will come in following years. For example, states will have to report 
the number of students and schools that participate in public school choice in their 
annual state Title I reports to USED. 

Some general information about local implementation can be provided, based on 
our state interviews, our review of state consolidated applications, and news articles 
about specific districts and states. In addition, some districts have compiled and made 
public various kinds of information about their use of the choice provisions. 

Districts have developed plans for implementing the choice requirements based on o q q o 
a number of factors that best fit their capacity and need. Some of the factors consid- ° 

ered include the number of children eligible for public school choice, the number of 61 

receiving schools that are not in school improvement status, the capacity of the receiv- 
ing schools to accommodate additional students, and the cost of providing transporta- 
tion. The following examples depict a range of implementation processes at the dis- 
trict level: 

□ In Montgomery County, Maryland, sending schools are paired with a particular set 
of receiving schools in the district, and students are transferred to the designated 
receiving schools (Montgomery County Public Schools, 2002). 

□ The New York City Department ofEducation centralized the public school choice 
option, lifting the borders from its school districts and allowing students to transfer 
within and across their home districts (New York State Department ofEducation, 

2002 ). 
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□ In St. Louis, the district takes into account the performance of potential receiving 
schools and considers as receiving schools only those with performance above a set 
bar, even if the school is not in improvement status. 

□ In Hamilton, Ohio, parents were allowed to select two schools from a list of seven 
receiving schools, and all parents got one of their top two choices. (The Center on 
Education Policy case study of this school district appears in Chapter 1.) 

Information from our state interviews, news reports, and other organizations sug- 
gests that in the early phases of implementation, the choice option is not being used 
extensively. According to the American Association of School Administrators (2002), 63 
62 out of 2,300 eligible students in Howard County, Maryland, applied for transfer, and 49 

o O o accepted the school they were assigned to. In Portland, Oregon, only 100 out of the 

3,000 eligible students applied. In Wichita, Kansas, two schools offered choice but as of 
September no parents had applied. In Fulton County, Georgia, 331 out of the 11,000 
eligible students from 10 schools applied and accepted transfers. In Clark County, 
Nevada, 109 of 4,500 eligible students were expected to move. In Montgomery 
County, Maryland, 101 of the 6,000 eligible decided to move. 

In some districts, however, enough students seem to be transferring to have an 
impact on the school that loses enrollments. At West Middle School in Uintah County, 
Utah, a rural school serving a largely Native American population, 60 students opted 
to transfer to a different school, despite a campaign mounted by parents to encourage 
other parents to keep their children in the local school. As a result, West Middle School 
lost 3.5 full time equivalent teachers, leaving math classes in the hands of a shop teacher 
and a science teacher (Sykes & Lynn, 2002). 

There are several possible reasons why the use of public school choice has been lim- 
ited so far in many districts. As already noted, some districts have delayed implementa- 
tion, so choice has not truly been made available. In other cases, parents may not have 
been well informed or may not be aware that public school choice is now available. 
Parents may have a strong familiarity with and commitment to their neighborhood 
schools, or a desire to improve the local school rather than leave it. For example, an offi- 
cial from a state in which only two students had transferred to another school felt that 
the local culture of community involvement in local schools could have made parents 
reluctant to exercise choice, especially if the receiving school is farther away. Some par- 
ents may also like the convenience of their child attending school close to home. 

In an effort to mount a public information campaign that would spread the word 
to parents about the NCLB choice option, the U.S. Department of Education award- 
ed a $600,000 grant to the Black Alliance for Educational Outcomes. This organization 
will develop a public information campaign to reach parents about the choices avail- 
able to them under the No Child Left Behind Act (BAEO, 2002). It is too early to pre- 
dict the effects of this campaign. Even with a national campaign, districts will still have 
to implement their own public information efforts. 



Current Issues, Problems, and Obstacles 

Based on state and local experiences so far, some issues, problems, and obstacles have 
emerged in implementing the choice provisions of the law. 
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□ Districts, especially large urban districts, are struggling with capacity and 

class size limits. 

In our interviews, some states reported that districts are struggling to place students who 
have requested transfers, because of limited capacity or class size constraints in the receiv- 
ing schools. Space in schools not identified as needing improvement is often limited, and 
districts find this demand difficult to accommodate, particularly in cases where large num- 
bers of students apply for transfers or many schools have been identified as needing im- 
provement. Los Angeles, for example, had 230,000 students who qualified for 100 avail- 
able spaces in fall 2002, while Baltimore had 30,000 students qualifying for 194 spaces 
(Holland, 2002). These examples occurred before the final regulations were released and 
USED had made clear that limited capacity cannot be used to deny students transfers to 
better-performing schools. To comply with the regulations, some school districts will 
have to scramble to make room in better-performing schools, adding teachers and classes 
and perhaps even adding mobile classrooms, split sessions, or other strategies used in over- 
crowded schools. The case study of St. Louis at the end of this chapter illustrates what 
one urban district is doing to try to make choice run smoothly. 

Before the final regulations were released, some states had passed legislation to limit 
the public school choice required by NCLB. In Illinois, the General Assembly approved 
legislation requested by the Chicago Board of Education to exempt magnet schools and 
crowded schools from the list of“receiving” schools (Catalyst Chicago, 2002). The final 
non-regulatory guidance makes clear that a district may not exclude magnet and spe- 
cial-focus schools from the list of choice schools. However, the admissions requirements 
for getting into these schools should also apply to transfer students. 



a Districts in rural and remote locations have practical issues with implementation. 

Some states in our survey reported this fall that serious logistical issues could make it dif- 
ficult for districts to implement public school choice options, even with a good faith ef- 
fort to comply with the law and regulations. For instance, some rural districts have only 
one school at each grade span and cannot make public school choice available to their 
students. Using NCES data, Alexander Russo (2002) calculated that among the more 
than 14,000 school districts in the nation, one in three has just a single school per grade 
level. In Kentucky, for example, there are six such districts. As advised by USED, some 
states have encouraged school districts facing this situation to provide supplemental edu- 
cation services to students in schools in their first year of improvement status, or to form 
cooperative agreements with neighboring school districts. For instance, one county 
school superintendent in Kentucky with only one school per grade span said that the 
district will offer supplemental instruction for students in the school found to be in need 
of improvement (Deffendall, 2002), In Hawaii, some students in geographically remote 
areas would have to transfer to another island to exercise choice. 



□ The public school choice option is being exercised for reasons other than 
the intent of the law. 

Some states officials that we interviewed indicated that some parents request transfers 
for reasons other than the intent of the law. For instance, parents may exercise choice 
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so their child can transfer to a school that is closer to their work. Or, students who are 
the highest achieving in a low-performing school could disproportionately request 
transfers. One state described a situation in which parents who had pulled their chil- 
dren out of public schools before NCLB have gone back to their neighborhood 
school to request a transfer for their child to attend a higher-performing public school, 
even though their child was not enrolled in the Title I school. The final regulations 
and guidance leave it to the states to define enrollment policies and how eligibility for 
enrolled students should be determined, but stipulate that transportation costs do not 
have to be paid for students who left a school in improvement prior to the enactment 
of NCLB. 

64 

Issues to Watch 

EXTENT OF USE 

In order for the federal government, states, and districts to have a clear picture of the 
extent of use of public school choice, the reasons why parents did or did not exercise 
this option should be studied. However, outside of collecting raw numbers on the chil- 
dren and schools using this option, there are currently no requirements for the district, 
state, or federal government to conduct detailed analyses. Tracking various trends can 
help state and federal policymakers better understand the issues surrounding public 
school choice and make decisions in the future informed by concrete information. 



IMPACT OF CHOICE ON STUDENT ACHIEVEMENT AND SCHOOL QUALITY 

A major question is whether choice is making a difference in student learning. To en- 
sure that the public school choice requirements are not just a band-aid solution that 
has little or no impact on achievement, policymakers and researchers should closely 
watch changes in the achievement level of students who exercise this option. Empirical 
evidence is needed to determine whether choice is contributing to the greater goal of 
improving student learning. 

Simply placing a low-achieving student in a higher-achieving school may not 
improve the performance of that student. Children who have been behind in the past 
and who have not received the supports needed to reach their academic potential will 
continue to remain behind unless they receive additional supports and/or services to 
help them address the problems that contributed to their poor performance. Claims 
that choice improved student performance should be backed up by evidence that shows 
what additional support(s) students received and to what extent the transfer has con- 
tributed to their achievement. 

A related question is what happens to the quality of education in a school, or for 
that matter across a district, if significant numbers of students request transfers from 
under-performing schools. Will the sending school improve, or will performance 
decline even more if many students leave, especially if those who leave are not the low- 
est achievers? What will happen to achievement in the receiving school if large num- 
bers of students enter who may not be prepared for a demanding curriculum? Research 
is also needed to answer these questions. 
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TRANSPORTATION COSTS 

Some districts are trying to keep costs down by limiting the schools to which students 
can transfer. In situations where single students choose to attend different schools a 
great distance from home, transportation costs for the school or district could be high. 
Districts will have to strategically determine the most cost effective way to handle 
transportation while complying with the law and regulations. For instance, Chicago 
has restricted transfers to within a three-mile radius (Catalyst Chicago, 2002). The final 
regulations and guidance note that school districts can establish such transportation 
zones as long as the choice option is made as broad as possible, but at a minimum in- 
cludes at least two choices. Additional experience with implementing the option will 
reveal whether transportation costs are a significant factor. 



PARENT REQUESTS THAT CANNOT BE MET 

In some states, there has been confusion as to what the public school choice option 
means. Some parents have made requests that cannot be met, such as requesting the 
choice of a private school. One state official interviewed said that disseminating cor- 
rect information to parents has been a tremendous task. 



Supplemental Education Services 

NCLB Requirements for Supplemental Services 

The federal Title I program has always allowed schools to provide additional instruc- 
tional services to children after school, during the summer, and on weekends. The No 
Child Left Behind Act marks the first time, however, that the federal law has included a 
mandatory supplemental education services requirement, and the first time that public 
school children are specifically allowed to receive services from providers outside the 
school system, paid for with federal funds. 

In addition to offering public school choice, Title I schools in their second or sub- o q Q o 
sequent years of school improvement status must offer supplemental educational serv- ° 

ices to students from low-income families. These services include tutoring and sup- 65 

plemental academic enrichment services, as shown in Box 3-B. These services will be 
paid for out of the same 20% set-aside of funds that school districts must reserve for 
choice programs. 

These supplemental services can be provided by a public or private school entity, 
educational agencies, a public or private institution of higher education, or a nonprof- 
it, for-profit or faith-based organization. Parents will choose the providers they want to 
serve their children from a list of state-approved providers that are in the area or that 
offer services via distance learning technology. The law calls on states to develop objec- 
tive criteria for selecting service providers that must include: 

1) A demonstrated record of effectiveness in improving student academic achievement; 
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BOX 3-B SUPPLEMENTAL EDUCATION SERVICES 

Supplemental education services should be: 

(1) In addition to instruction provided during the school day; 

(2) Specifically designed to — 

(i) Increase the academic achievement of eligible students as measured 
by the state's assessment system; and 

(ii) Enable these children to attain proficiency in meeting state academic 
achievement standards; and 

(3) Of high quality and research-based. 



Source: PL. 107-1 10, the No Child Left Behind Act 



2) Documentation that the instructional strategies used by the provider are high qual- 
ity, based on research, and designed to increase student academic achievement; 

3) Evidence that services are consistent with the instructional program of the school 
district and with state academic content standards; 

4) Evidence that the provider is financially sound; and 

5) An assurance by the provider that it will provide supplemental education services in 
accordance with applicable federal, state, and local health, safety, and civil right laws. 



Department of Education Activities 

On July 13 and 14, 2002, the U.S. Department of Education held a NCLB Supplemental 
Services conference in Washington, D.C., that was attended by representatives of state ed- 
ucation agencies and various education organizations. In the view of some states, the 
information provided by the Department at the July meeting on supplemental services 
was inadequate, as characterized by a story in the Title I Monitor (2002) with the head- 
line, “Supplemental Services Meeting Generates More Questions Than Answers.” 

On June 14, the Secretary released a letter to the chief state school officers that 
contained guidance about the supplemental services provision and reiterated that these 
opportunities were to be afforded to children beginning “this fall.” 
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The letter also offered some clarification to the law, including the following: 

□ School districts are not required to provide supplemental education services to stu- 
dents in a school that has been removed from the school improvement list based 
on 2001-02 assessment results. 

□ Parents, the school district, and the chosen provider must develop specific academ- 
ic achievement goals for the student, measures of student progress, and a timetable 
for improving achievement. 

□ If funds for supplemental services are limited, the lowest achieving students must 
be given priority. 

□ The list of state-approved providers must be updated at least annually. 

□ States may not require supplemental education service providers to hire only cer- 
tified teachers in order to be eligible. 

□ Regardless of whether the provider is a public, private or faith-based entity, all cri- 
teria for selecting providers must be applied consistently. 

□ Providers that are located outside the district or state are eligible to provide serv- 
ices if they meet the criteria used to approve all providers. 

□ Charter schools that receive Title I funds and are identified as being in need of 
improvement must offer supplemental education services. 

□ Supplemental education service funds can be used to transport students to and 
from approved providers. 

The final regulations issued on November 26, 2002, made several additional clari- 
fications to the supplemental services provision: 

□ The Department of Education has changed its earlier position about whether states 

can require supplemental service providers to document that their instructional 
strategies and services are grounded in “scientifically based research,’ ” as defined by 
NCLB. (As discussed in chapter 5 of this report, NCLB requires public schools to 
use instructional strategies based on scientific research in many federal programs.) v 
Although the Secretary’s June 14 letter said that providers should not be disquali- O 

fied on these grounds, the final regulations removed this interpretation, which 67 

means that states could impose such a requirement. 

□ The regulations require school districts to ensure that English language learners 
receive appropriate supplemental educational services and language assistance in 
the provision of those services. 

□ Supplemental educational services have to be consistent with, but do not have to 
meet the goals and objectives of, the Individual Education Program (IEP) for stu- 
dents with disabilities. Services should be in addition to other services received by 
students that are consistent with their IEPs and section 504 plans. 

□ The regulations prohibit school districts identified for improvement or corrective 
action from providing supplemental education services themselves. 
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On December 12, 2002, the Department released more detailed non-regulatory 
guidance on supplemental educational services. The guidance offered further clarifica- 
tions to the law, including the following: 

□ The state-approved list of supplemental education service providers must include a 
brief description of the services, qualifications, and demonstrated effectiveness of 
each provider. 

□ Faith-based organizations may become providers if they meet the requirements of 
the statute and regulations. 

□ Supplemental education service providers are not governed by the teacher quality 
requirements that apply to Title I schools. 

□ A school district may request an exemption from the state on all or part of the sup- 
plemental education service requirements if none of the providers can make their 
services available to the school district because of geographic constraints or by dis- 
tance learning or if the school district can provide evidence that it cannot provide 
these services. 



State Activities 

STATE CRITERIA 

Our review of state applications and our state interviews confirmed that all states used 
or planned to use the criteria from the legislation as a basis for rating and selecting 
providers. Some adapted the SEA Toolkit on Supplemental Educational Services in the No 
Child Left Behind Act, which was created by the Council of Chief State School Officers 
and the Education Quality Institute and funded by the U.S. Department of Education 
through the State Education Policy Network. This toolkit includes a sample applica- 
tion form for supplemental education service providers, a sample checklist for review- 
ing applications from prospective providers, recommendations for the process of ap- 
proving providers, communication issues for states to consider, and information 
resources such as websites. Some states used a combination of the law, the toolkit, 
and/or state-specific criteria in their selection process. 

The U.S. Department of Education did not provide any other model to help states 
prepare to implement the supplemental services provision before the 2002-03 school 
year began. 



STATE LISTS OF PROVIDERS 

When we interviewed states during the fall of 2002, 15 states had a list of approved sup- 
plemental service providers. Several states had interpreted the Secretary’s statement in his 
June 14 letter to mean that supplemental services must be made available this fall, al- 
though not necessarily by the beginning of the school year. In October, most schools 
were at least one month into the school year, and the majority of the states were still in 
some phase of the selection process. States were still developing a rubric for the applica- 
tion, waiting for the completed applications to be returned, assembling a task force to de- 
velop a process for selecting providers, reviewing applications, or waiting for the selected 
providers to confirm that they wanted to be included on the state-approved list. 
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In our interviews, the most common reason cited by states for this delay was that 
they were waiting until the federal government issued final regulations and guidance 
before selecting providers. Another reason was that in many states, relatively few Title I 
schools were in the stage of school improvement where they must offer supplemental 
education services. Some states, like Florida, did not have any schools in school 
improvement status, which meant that no students were eligible for supplemental serv- 
ices. Although these states are aware that they may have to produce such a list in the 
coming years, they have opted not to create a list this school year. Instead, they were 
focusing their time, money, and human resources on fully implementing the services 
and programs that are required for 2002-03. 

Finally, many states indicated that they were still waiting for the release of 2001-02 
assessment results. The U.S. Department of Education has given states the option to 
remove schools from the list of those identified for improvement if their 2001.-02 test 
results show achievement gains, but they do not have to use these data to add more 
schools to the list of those needing improvement. Some states have indicated that until 
their ‘01-‘02 data are reported, they cannot be certain which schools will need to pro- 
vide supplemental services. One state does not expect to receive ‘01-‘02 score results 
until the spring of 2003. Flowever, USED administrators have said in several interviews 
and written statements that waivers or delays in implementation will not be permitted. 
If a state elects not to use 2001-02 test results to add more schools this year, the regu- 
lations indicate that these data will be used in the following years to identify schools. 

Among those states that do have approved lists of providers, many states, including 
California, Maryland, New York, and Tennessee, have posted their lists on the state edu- 
cation agency website. The number of providers varies widely by state. Maryland had 
only two providers that were approved in the initial application process, while 
California had already approved 113 providers in September 2002 when our interviews 
were conducted. In some states, the lists for this school year will change, because the 
state has a rolling application process and will continue to accept applications and 
approve providers. In November 2002, Maryland released a second request for applica- 
tions and will add providers as they complete the process. 



LOCAL IMPACT 

Early estimates from cities like Los Angeles and New York suggest that far fewer stu- 
dents are seeking supplemental education services than might have been expected. In a 
survey conducted by the New York Times in November 2002, 10,000 of the 240,000 el- 
igible students in New York City had signed up with an outside provider (Goodnough, 
2002). Because so many states have not yet issued their lists, it is difficult to know 
whether this early trend will hold, or whether more parents and students will take ad- 
vantage of this option as information about it spreads. Unlike public school choice, 
use of third party supplemental service providers, selected by parents, is a new phe- 
nomenon in most of the country, and it will take a while for understanding of the con- 
cept to spread. 

One school district we spoke with was trying to stay ahead of the curve to provide 
supplemental services to its students. This district is located in a state that, like many 
others, had only a small number of schools required to offer supplemental services to 
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students in the 2002-03 school year. The state did not have the capacity to put together 
a list of providers by spring 2003, so with state permission, the district we interviewed 
in that state sent out its own request for proposals for supplemental service providers. In 
fall 2002, this district was working to set up contracts with several national and local 
service providers. Even though this district is required to provide supplemental services 
to the students in only one school, it is also making these services available to three other 
schools in the district (two that currently must offer choice and one that will likely be 
required to offer choice next year). Since transportation costs for school choice have 
proved to be low because of an agreement with the local public transportation admin- 
istration, the district has the funds to expand this option to these additional children. 
70 Rather than carry these Title I funds over to next year, the district has chosen to start 

^p o helping to improve the academic achievement of these students now rather than later. 



Issues to Watch 

Several issues should be watched to determine how the supplemental services provi- 
sions are implemented and what their effects are. 



HOW WILL STATES MONITOR THE EFFECTIVENESS OF SERVICES? 

NCLB requires states to monitor the quality and effectiveness of the services or pro- 
gram offered by approved providers. Since supplemental education services are only 
one part of the educational process that could affect learning and achievement, it will 
be difficult to identify whether a particular service actually improves student achieve- 
ment, and if so, to what extent. States face a challenge in figuring out how to isolate 
this part of the educational process. 



HOW ARE STATES DISSEMINATING INFORMATION TO PARENTS? 

Although the requirement in the law to inform parents of their options rests with local 
school districts, states should monitor how and if parents are informed about the avail- 
ability of supplemental services. In our interviews, many states were still in the process 
of developing their approved-provider list. It was unclear how this information will 
eventually get into the hands of parents. The New York Times reports that thousands of 
children in New York City will miss out on free tutoring that they qualify for under 
NCLB because the district did not give parents enough information or time to sign up 
(Goodnough, 2002) 



TO WHAT EXTENT ARE SUPPLEMENTAL EDUCATION SERVICES BEING USED? 

In future years, it should be easier to obtain data about how many students are receiv- 
ing supplemental services. In addition, research should look at the characteristics of 
students seeking these services. The legislation stipulates that students receiving supple- 
mental services should be from low-income families, but are they the lowest achieving, 
or a mix, and do the services that students seek match their academic need? Who are 
the providers? Are parents being adequately informed about the services offered by the 
different providers and their effectiveness? 
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Conclusion 

It is still early to predict what effect, if any, public school choice and supplemental edu- 
cation services will have on increasing student achievement. However, there are some 
key areas of implementation that should be monitored to help the federal government, 
states, districts, and schools identify the progress or lack of progress in providing these 



services. 
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CASE STUDY 

SCHOOL CHOICE IN ST. LOUIS 



The St. Louis Public Schools implemented school choice quickly, efficiently, and 
thoughtfully, and the district is already working to make next year even better. 

Working with a short timeline, the St. Louis, Missouri, Public Schools efficiently man- 
aged and carried out NCLB’s school choice provisions. Seventeen of the districts Title 
I schools failed to make adequate yearly progress for two consecutive years and were 
required to offer choice. On August 5, 2002, the St. Louis Public Schools sent out ap- 
o O°o proximately 7,000 school choice letters and applications to parents of students attend- 
ing these six middle and eleven elementary schools. Of 280 applicants, 125 students 
were offered places in six qualifying transfer schools, and parents were notified by Au- 
gust 23, 2002. Transfer students began the 2002-03 school year in their new schools, 
and as of September 24, 2002, 38 students had enrolled in transfer schools. 

Finding Space and Selecting Schools 

The two primary criteria for identifying schools, as transfer schools were academic per- 
formance and space. To help ensure that students would be less likely to go through the 
choice process again in the future, “challenge schools’’ — those deemed at risk of being 
placed in school improvement in the future — were not considered as potential transfer 
schools. Only solidly performing schools were chosen as transfer schools. 

Finding schools with available space was more difficult and was based on enrollment 
projections from the previous year, which led to one problem. After choice letters were 
sent out to parents, the district determined that there were a limited number of spaces 
in six elementary schools and none in middle schools. Thus, no middle school students 
were able to transfer this year even though they had been sent applications and letters. 

Finding places for special needs students was a high priority for the district. To 
make sure that choice was available for all special needs students, a few places were 
made available at elementary schools other than the six originally identified as transfer 
schools. 

Selecting and Placing Students 

The most time-intensive part of the school choice process was identifying and placing 
students. Since the number of spaces was limited, only low-income, low-achieving stu- 
dents were able to transfer. Students were ranked from highest to lowest priority in the 
following way. 

□ Students who scored below level in reading and mathematics and received free lunch. 

□ Students who scored below level in either reading or mathematics and received free 
lunch. 

□ Students who scored below level in reading and mathematics and received 
reduced-price lunch. 

□ Students who scored below level in either reading or mathematics and received 
reduced-price lunch. 
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After it was determined which students had the greatest need, the district then 
attempted to match students with schools that were close to where they lived to min- 
imize travel time for students. 

Placing English language learners and special education students also presented 
some additional challenges for the district. Letters and applications were translated into 
five different languages (Bosnian, Somali, Vietnamese, Spanish, and Farsi) and sent to 
parents of ELLs. Parents of ELLs requesting placement in choice schools were assigned 
schools with programs specific to the needs of ELLs. Special education students were 
placed based on their IEP, in consultation with the Special Education Office, to ensure 
that placement met IEP requirements. 

Aside from the logistical problems, placing students also caused some tension with 
parents, because they believed that they would be able to choose the school their chil- 
dren would be attending. While parents’ concerns were eventually addressed, the dis- 
trict had to spend a significant amount of time with them explaining the process. 

Lessons Learned and Concerns 

While school choice went smoothly this first year, for the most part, there are things 
the district would change for next year and would suggest for other districts just be- 
ginning the choice process. 

First, the district would advise other districts to form an NCLB implementation 
committee that includes parents and representatives from every department in the district, 
because every department, including transportation, ESL, and others, will be affected. 

Second, the district had thousands of returned letters because parent addresses had 
not been purged from the information system. Getting rid of old addresses not only 
cuts down on costs but also minimizes confusion on the part of parents whose students 
are no longer enrolled in choice schools. 

Third, districts should take advantage of whatever extra time they have to get the 
choice process started earlier. Because of the chosen timeline steps for AYP, Missouri 
will not be identifying any new schools under its accountability system in the coming 
year. Therefore, St. Louis already knows which schools will have to offer choice in the 
coming year. This helps them get a jumpstart on planning for next year, and they plan 
to send out letters to parents in January 2003 for choice in fall 2003. 

Impacts 

While long-term impacts of choice in St. Louis will not be seen for years to come, the 
district does have some first thoughts. The loss of funding in schools that had to offer 
choice was minor this first year because the number of students transferring was low. 
Most choice schools lost less than 10 students. In years to come, if more space is found 
in transferring schools, this may no longer be the case. 

While schools as a whole may not be greatly affected, the district was concerned 
that students whose parents opted for choice may lose out on the extra services pro- 
vided in the schools that are in need of improvement. These schools get extra funding 
and have additional programs that transfer schools do not. Therefore, while they believe 
that choice will increase performance for these students, it may not be as rapid as if they 
had stayed in their home schools. 



Source: Center on Education Policy school district interview 
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CHAPTER 4 




KEY FIMDIMGS 

□ At the time of our interviews, all the states we interviewed had or would soon 
have information on the number of teachers who are fully certified, but many 
states indicated that their data gathering systems will have to be revised to fully 
reflect the federal definition of "highly qualified" teachers. 

□ States have not made as much progress in gathering data about the qualifica- 
tions of paraprofessionals working in Title l-supported programs. Most states 
will need to start from scratch to develop systems to collect this information, or 
will rely on school districts to gather it. 

□ In interviews conducted for this report, most state personnel rated the written 
guidance issued by the U.S. Department of Education on the NCLB teacher 
quality provisions as "good" or "fair." But states gave the Department lower 
marks of "fair" or "poor" for the timeliness of this guidance. 

□ The federal definition of what constitutes a "highly qualified" teacher is tied to 

state certification and licensure requirements. Some experts have questioned 
the rigor of these systems for adequately preparing teachers and have called on o q q o 
states to make their teacher certification requirements more stringent. ° 

Therefore, the effect that NCLB teacher and paraprofessional qualification 75 

requirements could have on improving instruction will depend in part on both 

the rigor of states' teacher certification requirements and the evaluation instru- 
ments used to assess the knowledge and skills of current teachers and parapro- 
fessionals. 
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hrough the No Child Left Behind Act, the federal government is taking broad 
and unprecedented steps to raise the qualifications of the nations public 
school teaching force and of the instructional aides, also known as paraprofes- 



sionals, who work in Title I programs for disadvantaged children. 



NCLB requires states to ensure that all public school teachers who teach core aca- 
demic subjects are “highly qualified” by the end of school year 2005-06. (See Box 4-A 
76 for a list of the law s main deadlines affecting teachers and paraprofessionals.) The law 

o C> 0 essentially defines a highly qualified teacher as one who is fully certified and who 

demonstrates competence in the subject that he or she teaches by having completed 
certain academic coursework, passed a state examination, or successfully completed a 
rigorous state evaluation. The Act also requires most Title I paraprofessionals to have at 
least two years of college or to pass a state or local exam by January 2006. These feder- 
al requirements for teachers and paraprofessionals have already taken effect for two 
groups of newly hired personnel: teachers hired after the first day of school year 2002- 
03 to work in Title I programs; and Title I paraprofessionals who provide instructional 
services and were hired after January 8,2002 (the date the law was enacted). 

In both substance and scope, these requirements represent a major expansion of the 
federal role concerning teacher quality. Traditionally, states have determined the 
requirements for licensing and certifying people who want to teach in their states. With 
NCLB, the federal government, for the first time, has established minimum qualifica- 
tions for all teachers of core subjects, not just those whose salaries are paid for by a fed- 
eral program. The number of teachers affected will almost certainly be a very large share 
of those who teach core subjects of the nations 2.9 million public school teachers 
(National Center for Education Statistics, 2002). States still have the authority to set 
requirements for teacher preparation, certification, testing, and related areas, but the 
policies they establish will have to conform with federal law — which will mean a 
change of policies in most states. 

The new federal requirements will also have a substantial impact on paraprofes- 
sionals who provide instructional services. Under the old law, paraprofessionals sup- 
ported with Title I funds only needed a high school diploma or the equivalent, so the 
new law has significantly ratcheted up the minimum qualifications for this group — 
which in school year 1997-98, numbered almost 77,000 people (U.S. Department of 
Education, 1999a). Under NCLB, local school districts, in tandem with the states, are 
responsible for ensuring that their paraprofessionals meet these higher requirements. 
Many states or local school districts do not require Title I paraprofessionals to complete 
at least two years of college, nor do they have testing policies for paraprofessionals, and 
they will need to establish such requirements or policies as a result of the federal law. 

The new teacher and paraprofessional requirements are creating considerable chal- 
lenges for states as they implement NCLB. This chapter analyzes the issues faced by 
states and school districts in meeting these requirements and reports findings from our 
interviews with state department of education personnel. The chapter also highlights 
issues to watch as implementation proceeds during the next few years. 
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BOX 4-A DEADLINES FOR MEETING THE TEACHER AND 
PARAPROFESSIONAL REQUIREMENTS OF THE 
NO CHILD LEFT BEHIND ACT 

JANUARY 8, 2002 

All Title I paraprofessionals must have at least a high school diploma or its equivalent. 

All Title I paraprofessionals hired after January 8, 2002 (except those with non-instruc- 
tional duties such as those who serve as translators or conduct parent involvement ac- 
tivities) must have completed at least two years of higher education or have demon- 
strated their knowledge and skills through a test and other rigorous standards of 
quality. 

SCHOOL YEAR 2002-03 

All teachers hired after the first day of school year 2002-03 to work in a program sup- 
ported by Title I must meet the NCLB definition of "highly qualified." 

By May 2003, states must set annual measurable goals for each of their school districts 
and schools, including annual increases in the percentage of highly qualified teachers 
and the percentage receiving professional development to become highly qualified. 

School districts receiving Title I, Part A funds must begin filing annual reports on their 
progress, and the progress of their individual schools, in meeting their annual measura- 
ble goals. States must also submit an annual progress report to the U.S. Secretary of 
Education. 

School districts must inform parents of students attending schools receiving Title I 
funds that they may request information on the qualifications of their child's teachers. 
Also, parents of children in Title I schools must be informed in a timely manner, if their 
child has been assigned to or taught by a teacher who is not "highly qualified" for four 
or more consecutive weeks. 

JANUARY 8, 2006 

By January 8, 2006, all Title 1 paraprofessionals, regardless of when they were hired, 
must meet the NCLB qualifications of having completed at least two years of higher 
education or having demonstrated their knowledge and skills through a test and other 
rigorous standards of quality. Paraprofessionals who serve as translators or conduct 
parent involvement activities are exempted from these requirements. 

END OF SCHOOL YEAR 2005-06 

By the end of school year 2005-06, all public school teachers who teach core academic 
subjects must meet the NCLB definition of "highly qualified." 



Source: P.L. 107-1 70, the No Child Left Behind Act 



From the Capital to the Classroom 



Teacher Quality 

Background 

Before NCLB, the federal government played a limited role in improving the skills of the 
nation’s teaching force. Over the last 50 years, the major federal programs affecting teach- 
ers have included scholarship and fellowship assistance for teachers in training, student 
loan forgiveness for teachers working in high-need schools, professional development 
funds, and, starting in fiscal year 1999, federal aid to help states and school districts reduce 
class size. The National Science Foundation, the National Endowment for the Arts, and 

many other federal agencies also administer programs to improve teaching. 

78 

0 Through amendments to Title II of the Elementary and Secondary Education Act 

G° °G (ESEA), the No Child Left Behind Act continues federal aid to improve teachers’ skills 
and knowledge through high-quality professional development and to reduce class size 
by hiring more teachers. In FY 2002, Congress appropriated $2.85 billion for the state 
grants portion ofTitle II part A. Congress also appropriated an additional $18 million 
for a Title II program to help members of the military to become teachers, and anoth- 
er $35 million to prepare mid-career professionals from other fields and new college 
graduates to become teachers in high-need schools. 

It is the sections of NCLB often referred to as the teacher quality requirements that 
represent the real evolution of the federal role. These requirements are based on the 
assumption that students taught by fully certified teachers perform better academically 
than those taught by teachers who lack certification in the subject they are teach- 
ing. Several pieces of research corroborate this assumption. In one of the most com- 
pelling pieces, Dan Goldhaber and Dominic Brewer found that 12th grade students 
who were taught by certified teachers performed significantly better in mathematics 
than those students taught by teachers with private school certification or teachers 
uncertified in their subject area (Goldhaber & Brewer, 2000). 

The new provisions are intended specifically to rectify the problem of large numbers 
of students being taught by teachers with inadequate content knowledge and preparation 
in the subjects they are teaching. According to the National Commission on Teaching and 
Americas Future (1996), 23% of all secondary school teachers do not have a college 
major in their main field of teaching; among teachers who teach a second subject, 36% 
are unlicensed in the field and 50% lack a minor in the second subject matter. For spe- 
cific subject areas, a National Center for Education Statistics analysis of public high school 
teachers (2002) found that in the 1999-2000 school year, 21% of math teachers, 19% of 
English teachers, 17% of science teachers, and 39% of foreign language teachers did not 
have an undergraduate or graduate major in their main field of assignment. For advanced 
high school science subjects, the percentages without a major in their field were even 
higher: 51% of physics teachers, 54% of chemistry teachers, and 65% of geology/earth and 
space science teachers were under-qualified by this definition. 

These “out-of-field” teaching assignments, as they are called, are more prevalent in 
schools with large proportions of minority students and low-income students. In fact, 
the least experienced and least qualified teachers often teach students with the greatest 
educational needs. Based on an analysis of data from the U.S. Department of Education 
and other sources, the Education Trust concluded that 34% of secondary school-level 
courses in high-poverty schools were taught by teachers without at least a college 
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minor in their subject, compared with 19% of secondary courses in low-poverty 
schools (Jerald, 2002). Because federal Title I funds flow to schools with large propor- 
tions of low-income students, the qualifications of teachers in these schools have been 
a particular concern of the Congress and the Administration. 

These problems of out-of-field teaching persist because many school districts, espe- 
cially urban districts, have difficulty attracting and keeping highly qualified teachers. 
Many rural schools have similar difficulties, or they have such small enrollments that 
they must assign teachers to teach multiple subjects. The choice in these settings often 
comes down to hiring an under-qualified teacher versus hiring no teacher at all, or reas- 
signing a teacher to a subject outside of his or her field. It is too early to tell how the 
teacher quality provisions of NCLB will impact school districts that face these choic- 
es, but it could be that NCLB will exacerbate an already difficult situation. 



Teacher Quality Requirements of NCLB 

Some of the significant new teacher requirements already mentioned are contained in 
the NCLB amendments to Title I of the Elementary and Secondary Education Act. 

States that receive Title I funds — as all states do — must ensure that all of their public 
school teachers who teach core academic subjects (English, reading or language arts, 
mathematics, science, foreign languages, civics, government, economics, arts, history, 
and geography) are highly qualified by the end of school year 2005-06. The law’s crite- 
ria for what constitutes “highly qualified” vary (the specific criteria are described in 
Box 4-B), but generally speaking, they require teachers to be fully certified and to 
demonstrate their knowledge and skills by either having taken sufficient academic 
coursework in their field, passing a state test, or successfully completing a rigorous state 
evaluation. At the beginning of each school year, schools receiving Title I assistance 
must inform parents of students attending such schools that they may request informa- 
tion on the professional qualifications of their child’s teacher. 

The new federal criteria are effective immediately for public school teachers who 
were hired the day after the first day of school year 2002-03 (which varied, depending 
on the school districts schedule) and are teaching core subjects in programs supported 
by Title I, Part A funds. (Part A, the largest component of Title I, authorizes grants to ° 0 o 0 ° 
school districts.) In draft regulations for NCLB issued in August 2002, the U.S. 
Department of Education made an important clarification. The regulations specified 
that the federal teacher requirements apply not just to newly hired teachers in Title I 
“targeted” assistance programs (those that operate as separate programs within a school 
and serve only low-achieving children), but also to newly hired teachers of core aca- 
demic subjects inTitle I “schoolwide” programs (those that use Title I funds to improve 
instruction for all children throughout an entire school; only schools with at least 40% 
of the students from low-income families are eligible to operate schoolwide programs). 

States must develop a plan outlining how they will meet the federal teacher require- 
ments by the end of the school year 2005-06. The plan must establish annual measurable 
goals for each school district and school in the state; these goals must include an annual 
increase in the percentage of highly qualified teachers in each school district and school, 
and an annual increase in the percentage of teachers receiving high-quality professional 
development aimed at making them highly qualified and successful classroom teachers. 
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BOX 4-B FEDERAL DEFINITION OF "HIGHLY QUALIFIED TEACHER" 



The No Child Left Behind Act requires that all public school teachers teaching core aca- 
demic subjects be "highly qualified." As shown below, this term has different meanings 
for current versus new teachers and for elementary versus middle or high school teachers. 

NEW TEACHER - ELEMENTARY SCHOOL 

A public elementary teacher who is new to the profession is considered to be highly 
qualified under the No Child Left Behind Act if he or she has: 

1) Obtained full certification or passed the state's licensing exam and holds a license to 
teach; AND 

2) Has not had certification or licensure requirements waived on an emergency, tempo- 
rary or provisional basis; AND 

3) Holds at least a bachelor's degree; AND 

4) Has demonstrated, by passing a rigorous state test, subject matter knowledge and 
teaching skills in reading, writing, mathematics, and other areas of the basic school 
curriculum. 



NEW TEACHER - MIDDLE AND HIGH SCHOOL 

A public middle or high school teacher who is new to the profession is considered to be 

highly qualified under the No Child Left Behind Act if he or she has: 

1) Obtained full certification or passed the state's licensing exam and holds a license to 
teach; AND 

2) Has not had certification or licensure requirements waived on an emergency, tempo- 
rary or provisional basis; AND 

3) Holds at least a bachelor's degree; AND 

4) Has demonstrated a high level of competency in each of the subject matters he or 
she teaches by: 

A) Passing a rigorous state academic subject test in each of the subjects he or she 
teaches; this subject matter knowledge could be assessed either with a state-re- 
quired certification or licensing test, or with a separate test in each academic 
subject that the teacher teaches; OR 

B) Successfully completing an academic major, graduate degree, coursework equiv- 
alent to an undergraduate major, or advanced certification or credentialing in 
each of the academic subjects he or she teaches. 

CURRENT TEACHERS - ELEMENTARY, MIDDLE AND HIGH SCHOOL 

A public school teacher who is not new to the profession is considered to be highly quali- 
fied under the No Child Left Behind Act if he or she has: 

1) Obtained full certification or passed the state's licensing exam and holds a license to 
teach; AND 
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2) Has not had certification or licensure requirements waived on an emergency, tempo- 
rary or provisional basis; AND 

3) Holds at least a bachelor's degree; AND 

4) Meets the appropriate criteria outlined above for teachers new to the profession, in- 
cluding the option for passing a rigorous state academic subject test; OR 

5) Has demonstrated competence in all the academic subjects in which the teacher 
teaches based on a high objective state uniform standard of evaluation. 



CHARTER SCHOOL TEACHERS 

Public school teachers in charter schools who are in compliance with the state's charter 
school law regarding certification or licensure of such teachers are considered "highly 
qualified." 



ALTERNATIVELY CERTIFIED TEACHERS 

A teacher who obtained certification through an alternative route is considered to meet 
the state certification requirement. However, the final regulations issued by the U.S. De- 
partment of Education clarify that a teacher in an alternative route program is highly 
qualified if he or she is making satisfactory progress toward full certification. Such teach- 
ers must receive high quality, sustained professional development before and while 
teaching; participate in a program of intensive supervision; and assume functions as a 
teacher for no more than three years before receiving full certification. 



Source: PL. 107-1 10, the No Child Left Behind Act 
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School districts that participate in Title I must also develop plans for meeting the federal 
teacher criteria by 2005-06. Beginning in school year 2002-03, both the states and school 
districts must annually report on their progress in meeting their measurable goals. School 
districts must also require principals ofTide I schools to annually attest that their schools 
are in compliance with the teacher and paraprofessional qualification requirements. 

The No Child Left Behind Act also authorizes federal funding to increase the num- 
ber of highly qualified teachers. Districts must use at least 5% of their Title I, Part A grant 
for professional development aimed at helping teachers become highly qualified (which 
is not a new source of funding but a redirection of existing funds). A school that has been 
^ identified as needing improvement must reserve 10% of its funds to use for professional 

development activities. Also the revised version of Title II of ESEA authorizes federal 
0 0°0 0 grants to states to train and recruit highly qualified teachers and principals. 

Title II also spells out what happens to school districts that fail to make sufficient 
progress in raising teacher qualifications. If a state determines that a school district has 
failed to make progress toward its annual teacher quality goals for two consecutive years, 
the district must develop an improvement plan addressing the obstacles that prevented 
it from reaching its goals. The state must also provide technical assistance to the district. 
If the school district fails to make progress toward its teacher quality goals for three con- 
secutive years, and also fails to make adequate yearly progress in raising student achieve- 
ment for three consecutive years (see chapter 2 for an explanation of the student 
achievement requirements), then the district must work out an agreement with the 
state about how to use its Title II funds for professional development activities and 
develop strategies that will help it reach its teacher goals. 



U.S. Department of Education Actions 

On June 6,2002, the U.S. Department of Education issued draft guidance for the “Im- 
proving Teacher Quality State Grants” program authorized by Title II. On August 6, 
2002, the U.S. Department of Education issued draft regulations for the teacher quality 
provisions included in Title I. 

The final Title I regulations, issued by the U.S. Department of Education on 
November 26, 2002, included an important clarification with regard to teachers who enter 
the teaching profession through alternative routes. The final regulations clarified that alter- 
native certification programs should not become “vehicles for granting long-term waivers 
of certification requirements” and specified that teachers in alternative route programs 
should receive sustained, high-quality professional development, participate in a program 
of intensive supervision, assume functions of a teacher for no more than three years before 
receiving full certification, and demonstrate satisfactory progress toward full certification. 

On December 19, 2002, USED issued updated guidance on the Teacher Quality 
State Grants program. Some of the issues raised in this chapter may be addressed by this 
guidance, but we were not able to include a discussion of the updated guidance because 
of publication deadlines. 

As shown in Figure 4-A, more than three-fourths (77%) of the state personnel we 
interviewed rated the quality of the written guidance given by the U.S. Department of 
Education on teacher issues as “fair” or “good.” (These questions were asked before the 
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FIGURE 4-A STATE RATINGS OF QUALITY OF WRITTEN 
FEDERAL GUIDANCE ON TEACHER ISSUES 



Excellent 5% 




Number of states responding = 43 

Note: Totals exceed 100% due to 
rounding. 



Source: Center on Education Policy state interviews 



FIGURE 4-B STATE RATINGS OF TIMELINESS OF WRITTEN 
FEDERAL GUIDANCE ON TEACHER ISSUES 




Source: Center on Education Policy state interviews 
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final regulations came out.) But as Figure 4-B illustrates, these state officials gave the 
Department lower marks on the timeliness of its guidance on these issues, with 81% 
rating it as “fair” or “poor” The majority of our interviews with the states took place 
before a key set of regional meetings held by the U.S. Department of Education in 
October, so we do not have information about whether states found these meetings 
helpful in implementing the teacher quality provisions. 




State Activities 

DATA COLLECTION 

84 

A critical question on the minds of policymakers, educators, and the public is how 
q o o q many current teachers are not highly qualified. For states to be able to answer this ques- 
tion and set measurable yearly goals for improving teacher quality, they must have a 
system for keeping track of the qualifications and assignments of public school teach- 
ers. All of the states we interviewed either already had a system for tracking school per- 
sonnel in the state or were in the process of developing such a system. Some states have 
created electronic systems for retrieving this information from school districts, while 
others conduct periodic paper-and-pencil surveys of school districts. 

Many of the states with existing teacher tracking systems said they would have to 
revise their existing systems to align them better with the requirements of the No Child 
Left Behind Act. Some states have already started making modifications, while others 
said they were waiting for more guidance (and hopefully more clarification) from the 
U.S. Department of Education about such issues as how to handle long-term substitute 
teachers and whether teachers with an “emergency” certificate that allows them to 
teach while pursuing full certification through an alternative route can be considered 
“alternatively” certified for purposes of NCLB. 

Anecdotal evidence suggests that in some states and districts, the numbers of teach- 
ers not meeting the definition of “highly qualified” are large. Illinois had estimated that 
about 25,000 out of 130,000 teachers in the state are teaching without full credentials 
or in subjects for which they are not certified (Sack, 2002). A report of the Maryland 
Department of Education estimated that 10% of newly hired teachers, or more than 
5,300 teachers, were hired with provisional certificates, meaning they had not yet passed 
the necessary state exams or coursework (Labbe, 2002a). In Philadelphia, by one report, 
more than half of the 600 new teachers hired this year were not fully certified 
(Mezzacappa, 2002). 

States are also working to meet federal requirements for reporting and accounta- 
bility. We asked states if they had established a reporting system for school districts in 
order to determine compliance with NCLB s educator quality provisions. Forty-one of 
the 44 states that responded to our question indicated that they have established a sys- 
tem for reporting information on teacher qualifications, or are in the process of devel- 
oping one. We also asked states how they are ensuring that all new teachers hired after 
the first day of this school year to work in programs supported with Title I meet the 
definition of highly qualified. The 45 states that responded to our question reported a 
variety of strategies for monitoring compliance with this provision, including review- 
ing state licensure information to determine which schools were in compliance and 
which were not, checking compliance during state audit visits to school districts, or 
working with school districts to assist them in meeting the requirements. 
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STATE LICENSING AND TESTING REQUIREMENTS 

The No Child Left Behind Act requires teachers new to the profession to be fully cer- 
tified and to demonstrate appropriate knowledge and skills by passing a state test or, in 
the case of secondary school teachers, by having at least an academic major in the sub- 
ject being taught. According to Education Week , 29 states and the District of Columbia 
require beginning teachers to pass exams of both basic skills and subject matter knowl- 
edge. Twenty- nine states and the District of Columbia require high school teachers to 
have a major in the subject matter they are teaching, and four of those and the District 
of Columbia also require middle school teachers to have a major in their subject as- 
signment (Olson, 2002). 

In some states, the federal requirement for middle school teachers to have either a 
major in their assigned subject or pass a knowledge test in their subject is posing prob- 
lems. While most middle schools encompass grades 6 though 8, some include grade 5, 
which is typically taught by teachers with a general elementary school endorsement, 
rather than a subject matter specialty. Some middle schools are structured around 
departments, while others assign one teacher to teach reading and social studies and 
another to teach mathematics and science — a situation that could require teachers to 
be certified in multiple subjects under the new law. States that require only a general 
endorsement for middle school teachers are grappling with how they can carry out the 
NCLB requirements for this level of schooling. At press time, the U.S. Department of 
Education had not provided any written guidance on this issue. 

When we interviewed state education personnel, we asked them about any recent 
or anticipated changes in the state s certification requirements. Thirty-two of the 44 
states responding to this question reported that their states had made changes to their 
certification requirements in the last two years or will make them soon. In nearly all 
cases, the changes were meant to strengthen the certification process, either by having 
more rigorous requirements or by clarifying an earlier vague system. We do not know 
if recent changes made to state certification systems were in response to NCLB or if 
other conditions existed that caused the states to take action. Beginning on the next 
page, a case study of Kansas discusses how changes to that state s licensure system has 
helped to prepare it to implement the teacher requirements of NCLB. 

STATE STRATEGIES TO DEVELOP HIGHLY QUALIFIED TEACHERS 

According to our review of state consolidated applications, states are planning a variety 
of approaches to ensure that teachers meet the federal requirements. Some of the most 
common or interesting strategies are shown in Table 4-A. A number of states plan to 
adopt new certification/licensure systems, including performance-based systems, or to 
change their existing ones. Several states already have or are putting in place alternative 
routes to certification, and some states intend to provide financial assistance to teachers 
to help with certification costs. One state was developing strategies to improve the 
passing rates of lower-performing demographic groups on an entry-level teacher com- 
petency exam; and another planned to phase out provisional certification altogether. 
Many states intended to improve their support systems for teachers, including mentor- 
ing for new teachers or special programs to retain teachers in high-poverty areas. Final- 
ly, numerous states planned to form partnerships with higher education institutions, 
businesses, and other entities to better prepare under-qualified teachers. It should be 
noted that some states listed multiple approaches. 
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CASE STUDY 

TEACHER QUALITY IN KANSAS 



Kansas's work in changing its teacher licensure system has positioned it well to 
implement NCLB's teacher quality provisions. 

The emphasis of No Child Left Behind on improving teacher quality has proved to be 
a good fit with Kansas’s own education reform efforts. Improving teacher quality has 
86 been a long-term goal for Kansas. The state has spent the last 10 years revising its 

O teacher licensure system, and on July 1, 2003, a new performance-based licensure sys- 

O O tern will be instituted in the state. In addition to the state’s own commitment to im- 
proving teacher quality, Kansas has sought help from the federal government in this 
task and is currently in its third year of a Higher Education Act Title II Teacher Quali- 
ty Enhancement Grant. 

Kansas’s work has given the state a jumpstart in the implementation of the NCLB 
teacher quality provisions in a variety of areas. Although much of this work was not in 
response to NCLB, it does address many of the requirements of the law. 



More Rigorous Licensure 

In hopes of producing more highly qualified teachers, the state’s new licensure system 
will be far more rigorous than the previous one. First, the state has changed the types 
of assessments teachers must pass to be licensed in the state. Teachers will continue to 
take a pedagogy test, Principles of Learning and Teaching, but they will also be required 
to take Praxis II content area tests and a performance assessment. The performance as- 
sessment will be a portfolio based on professional education standards in the state. Also, 
15 of 22 teacher education programs in the state currently require a subject area major 
for their students, and the other 7 require content coursework equivalent to a major. 



Current Teacher Qualifications 

Kansas’s current data collection system for teacher qualifications, the Certified Person- 
nel Report, will have to be modified to gather the information required under NCLB. 
However, the state believes that these changes are minor and that current data are close 
to meeting the requirements of NCLB. 

For the most part, the state believes that its teaching force is in “good shape.” 
Approximately 95% of teachers in Kansas are appropriately certified for the teaching 
assignments that they hold. However, the state does have some concerns. Three percent 
of the state’s teachers are adding endorsements and are provisionally certified in those 
new areas, meaning that they hold one certification and are teaching in an area where 
they are more than halfway done with the requirements for this second certification. 
The state has set the bar for provisional certification much higher than in other states 
and is concerned about the implications of this strict definition under NCLB. 

The state is much more concerned about the number of waivers it has granted to 
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teachers. Due to some teacher shortages, especially in special education but also in 
math, science, music, and foreign languages, the state has granted waivers for licensed 
teachers who have just begun a second certification and are teaching in that area. 
Ensuring that these teachers meet the NCLB requirements may be the state s most dif- 
ficult implementation problem. 



Veteran Teachers 

The state has yet to choose a “rigorous standard of evaluation” for veteran teachers to 
demonstrate competence in teaching. In early December, the state will convene a group 
of Kansas master teachers, certification experts, and National Education Association 
(NEA) representatives to discuss options. 

The NEA has offered some suggestions, but the state has not felt that they have 
been rigorous enough. A pencil and paper test has also been discussed. If this is the route 
the state takes, it would like to develop its own tests for this purpose, so the tests would 
reflect that veteran teachers were trained in a different era. However, the cost of doing 
so may be prohibitive. 



Recruiting and Retaining Teachers 

The state agrees with the teacher quality goals of NCLB. One state official said, “We 
can’t argue with the intent [of NCLB] ” However, the hard part will be “the logistics 
of financing this.” Kansas currently offers less competitive salaries than many sur- 
rounding states, and coming budget cuts in the state will likely compound the prob- 
lem. The state does not expect to offer raises for teachers next year, and several dis- 
tricts do not offer health insurance to their teachers. On top of these financial 
considerations, the state is concerned that the additional requirements for new and 
veteran teachers under NCLB may make it even more difficult to get people to enter 
and remain in the field of teaching. 




Source: Center on Education Policy state interviews 
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TABLE 4-A SELECTED STATE APPROACHES TO 

DEVELOP HIGHLY QUALIFIED TEACHERS 



ACTIVITY 

IMPROVED ASSESSMENT AND EVALUATIONS FOR TEACHERS 
Examples: 

State-administered assessment for teachers 

Alignment of professional development and teacher evaluation with state standards 

REFINEMENTS OF CERTIFICATION AND LICENSURE 
Examples: 

Alternative certification routes 

New licensing system based on performance 

Financial assistance to help teachers with certification costs 

New procedures for reviewing and accrediting teacher preparation programs 

Phasing out of provisional certification 

Strategies to increase passing rates of lower-performing demographic groups on 
teacher professional entry exam 

FINANCIAL AND OTHER INCENTIVES 

Examples: 

Incentives to recruit teachers such as housing, travel assistance, scholarships, loan 
forgiveness 

Incentives for teachers to pursue national board certification 
Higher teacher salaries 

PROFESSIONAL DEVELOPMENT AND TRAINING 
Examples: 

Professional development models for specific academic areas 

Financial assistance for teachers to participate in professional development 

RECRUITMENT AND HIRING 

Examples : 

Recruitment through websites, clearinghouses, job fairs, other databases 

SUPPORT AND MENTORING FOR TEACHERS 
Examples: 

Mentor or support systems for new teachers 
Support systems for teachers in high-poverty areas 

Source: Center on Educat/on Policy review of consolidated state applications 
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Quality of Paraprofessionals 

Background 

The Congress and the Administration raised the qualifications required for paraprofes- 
sionals providing instructional services in Title I-supported programs because of con- 
cerns that classroom aides were being widely used to provide direct instruction to stu- 
dents. Many of these aides do not receive adequate supervision from teachers, and many 
lack the appropriate educational background to provide primary instruction for these 
students even for a limited time. 

According to the Congressional Research Service, about half the staff hired with 
Title I, Part A funds are paraprofessionals, and their salaries make up about 15% ofTitle 
I, Part A costs (Riddle, 2002). The use of paraprofessionals is common in Title I pro- 
grams in high-poverty areas. In the 1997-98 school year, 84% of principals in high- 
poverty schools reported using paraprofessionals, compared with 53% of principals in 
low-poverty schools. Nearly all Title I paraprofessionals provide instruction to students: 
98% ofTitle I aides reported that they either taught or helped teach students, and over 
50% reported that these instructional duties were carried out without a teacher being 
present (USED, 1999a). 



Paraprofessional Requirements of NCLB 

The No Child Left Behind Act has raised the qualifications expected of most parapro- 
fessionals working in Title I. Under the previous law, Title I paraprofessionals, except 
those acting as translators, had to have a high school diploma or its equivalent or had 
to obtain one within two years of being employed. Under NCLB, local school districts 
had to ensure that as of January 8, 2002, all Title I paraprofessionals, including transla- 
tors, had at least a high school diploma or its equivalent. In addition, districts must en- 
sure that paraprofessionals hired after this date meet the more stringent requirements 
described in Box 4-C, which include completing two years of higher education, or ob- 
taining an associate s degree, or meeting a “rigorous standard of quality” and demon- 
strating knowledge by passing a state exam. By January 8, 2006, all paraprofessionals 
working in Title I-supported programs must meet the more stringent requirements, 
except those who do not carry out instructional duties such as translators and parapro- 
fessionals who work solely on parental involvement activities. Also, paraprofessionals 
cannot provide direct instruction to students unless they are under the direct supervi- 
sion of a teacher. 



U.S. Department of Education Activities 

On April 28, 2002, the U.S. Department of Education sent a letter to chief state school 
officers briefly informing them of the requirements of the No Child Left Behind Act 
regarding paraprofessionals. The draft regulations issued in August 2002 made several 
important clarifications to the paraprofessional requirements, which remained un- 
changed in the final regulations issued in November 2002. 

The regulations elaborate on what it means for a paraprofessional to be supervised 
by a teacher. A paraprofessional is considered to be working under the supervision of a 
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BOX 4-C NEW REQUIREMENTS FOR PARAPROFESSIONALS 



The No Child Left Behind Act contains several new requirements regarding the qualifica- 
tions and duties of paraprofessionals working in any program supported by Title I funds. 

HIGH SCHOOL DIPLOMA 

Immediately after the legislation was enacted (January 8, 2002), all Title I paraprofession- 
als, regardless of their hiring date, had to have at least a high school diploma or its equiv- 
alent. Under the prior law, paraprofessionals without a high school diploma or its equiva- 
lent could work as translators. 



POSTSECONDARY EDUCATION OR ASSESSMENT 

Title I paraprofessionals hired after January 8, 2002, except for those with non-instruc- 
tional duties, such as those who serve primarily as translators or whose only duty is to 
conduct parent involvement activities, must have: 

□ Completed two years of study at an institution of higher education; OR 

□ Obtained at least an associate's degree; OR 

□ Met a rigorous standard of quality and demonstrated, through a formal state or 
local assessment, that they have knowledge of reading, writing, and mathemat- 
ics (or of reading readiness, writing readiness, and mathematics readiness) and 
that they have the ability to help instruct students in these subjects. 

All Title I paraprofessionals, except for those who carry out non-instructional duties, must 
meet these requirements by January 8, 2006. 



INSTRUCTIONAL DUTIES 

Title I paraprofessionals may provide instructional services to a student only if the para- 
professionals are working under the direct supervision of a teacher. 



Source: RL. 107-110 , the No Child Left Behind Act 



teacher “if the teacher plans the paraprofessional’s instructional activities and evaluates 
the achievement of the students with whom the paraprofessional works.” The parapro- 
fessional and the teacher must be working in “close physical proximity,” and the teacher 
must be “highly qualified” The regulations also make clear that the new qualifications 
requirements apply not only to paraprofessionals working in Title I targeted assistance 
programs, but also to those working in Title I schoolwide programs. Finally, the regu- 
lations clarify that Title I, Part A funds reserved for teacher professional development 
can also be used to help paraprofessionals meet the new requirements. 

On November 7, 2002, the U.S. Department of Education issued draft non-regu- 
latory guidance on the Title I paraprofessional provisions. In addition to clarifying some 
basic issues, such as outlining what constitutes a “newly hired” paraprofessional, and 
explaining what “two years of study” at an institution of higher education means, the 
draft guidance shed light on two areas where states and school districts had sought clar- 
ification. 

Several states we interviewed wanted clarification from USED about how to han- 
dle aides funded through special education who work with children with disabilities in 
Title I schoolwide programs. The draft guidance states that these aides must meet the 
requirements for Title I paraprofessionals if they provide instructional services, regard- 
less of the source of funds that support their salaries. However, if a special education 
aide in a Title I schoolwide program provides only personal care services, then the aide 
is not considered to be a paraprofessional under the Title I program so the requirement 
does not apply. 

The guidance states that the purpose of the paraprofessional assessment is to have 
paraprofessionals demonstrate that they have “the ability to assist in instructing students 
and are competent in required instructional techniques and academic content areas.” 
The exams, according to the guidance, should be sufficiently rigorous to evaluate para- 
professionals at a level equal to the second year of college. The exams do not have to 
be “paper and pencil” tests, but there must be evidence that the tests are “valid and reli- 
able” and “the results must be documented.” However, the guidance makes clear that 
the U.S. Department of Education does not have the authority to review or approve 
the proposed state or school district assessments for paraprofessionals. 

State Activities 

Our interviews with state education personnel indicate that states are not as far along 
in determining the qualifications of current Title I paraprofessionals as they are with 
teacher qualifications. Of the 46 states that answered our question, 9 states reported 
that they had complete information on the qualifications ofTitle I paraprofessionals, 
while 13 states had some information and 7 were in the process of developing a system 
to collect information. Four states will require school districts to collect the informa- 
tion and two states reported that they will collect the information during state compli- 
ance audits. Two states were waiting for more information from USED regarding para- 
professional qualifications before they proceeded to collect the information, and nine 
states responded that they did not have the information and did not indicate how or if 
they planned to collect it. 
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We asked states what they were doing to make sure that Title I paraprofessionals 
hired after January 8, 2002, met the new requirements. Of the 45 states that answered 
our question, 14 said they had given school districts information about the require- 
ments and were leaving it up to local officials to comply Eleven states said they will 
monitor compliance with these provisions through annual state audit visits. Other states 
reported a variety of means to monitor compliance, including reviewing information 
from the state’s school personnel survey and requiring assurances on school district 
applications that they are in compliance with the provision. One state, which has been 
granted authority to waive a range of federal requirements through the federal “Ed- 
Flex” program, said it would monitor compliance by noting which school districts 
92 sought waivers of the paraprofessional requirements. 

0 0 ° 0 0 Although systematic data are not available about the current qualifications of para- 

professionals, anecdotal evidence suggests that in some districts, large numbers ofTitle 
I paraprofessionals will be affected by the new requirements. In Denver, for example, 
the superintendent estimated that about 2,500 instructional aides did not meet the 
NCLB requirements (Labbe, 2002b). In Philadelphia, the new requirements are pro- 
jected to affect more than 2,000 paraprofessionals (Mezzacappa, 2002). 

Despite the apparent need, less than half of the nation’s school districts had some 
type of system in place to encourage paraprofessionals to pursue more education. 
According to U.S. Department of Education data from 1997-98, 38% of school districts 
provided a career ladder for paraprofessionals to move into teaching careers; 33% 
offered funding for higher education classes; 22% provided release time for paraprofes- 
sionals to attend or study for college classes; and 2% made funding available for para- 
professionals to obtain a high school diploma (USED, 1999). 

Our review of the state consolidated applications shed light on the activities that 
states plan to undertake to upgrade the skills of paraprofessionals. Table 4-B shows some 
of the most common or noteworthy options being planned. For example, several states 
plan to collaborate with higher education institutions or other states in their region to 
develop programs to strengthen the qualifications of paraprofessionals. Some states 
intend to develop or refine state certification requirements for paraprofessionals. One 
state plans to offer electronic training modules for paraprofessionals, while another plans 
to provide school districts with a list of competencies they can use in hiring parapro- 
fessionals. 

Although we did not directly ask states whether they would be developing or 
adopting an assessment to determine if a paraprofessional is meeting a rigorous standard 
of quality, some states volunteered that they intended to do this. In their consolidated 
applications, 17 states said that they already use or plan to adopt commercial assessments 
for paraprofessionals. Several states participated in the pilot of a new assessment for 
paraprofessionals called ParaPro, developed by the Educational Testing Service. One 
state indicated that it would have paraprofessionals take the Praxis I exam, a test devel- 
oped by the Educational Testing Service to assess the general knowledge and skills of 
teachers. Some states said they were still considering various options, while others said 
they were leaving it up to school districts to determine which assessment to use, since 
the law clearly states that the exam can be a state or local assessment. 
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TABLE 4-B SELECTED STATE STRATEGIES TO IMPROVE 

THE QUALIFICATIONS OF PARAPROFESSIONALS 



ACTIVITY 

INFORMATION DISSEMINATION AND GATHERING 
Examples : 

Meetings and sessions discussing new requirements 

Sample communication on how to inform affected paraprofessionals 

ASSESSMENTS 

Examples: 

State-level assessment for paraprofessionals 

Guidance to districts in selecting assessments for paraprofessionals 

CERTIFICATION 

Examples: 

State certification requirements for paraprofessionals 
Incentives for paraprofessionals seeking teacher certification 

TRAINING AND PROFESSIONAL DEVELOPMENT 
Examples: 

Professional development and training programs to help paraprofessionals meet 
requirements 

E-learning training modules 

Programs to encourage people to become paraprofessionals 

COLLECTION AND MONITORING OF QUALIFICATIONS INFORMATION 
Examples: 

List of competencies for school districts to use in hiring paraprofessionals 
Annual performance targets for meeting paraprofessional requirements 

PARTNERSHIPS AND COLLABORATIONS 
Examples: 

Collaborations with higher education institutions to provide necessary courses for 
paraprofessionals 



Source: Center on Education Policy review of state consolidated applications 
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Issues to Watch 

Although the federal law is fairly explicit about the qualifications that teachers and 
paraprofessionals should have, states still have latitude to design approaches to reach 
these goals. Whether the new teacher and paraprofessional requirements will result in 
better instruction for children could rest in large part on how states and school districts 
interpret, apply, and monitor the new criteria. It also depends on how vigorously the 
U.S. Department of Education enforces the requirements. Secretary Paige, with sup- 
port from key leaders on Capitol Hill, has indicated that the Bush Administration and 
his department intend to strictly enforce NCLB. Therefore, both state and local lmple- 
94 mentation and federal enforcement are key issues to watch in the future. 

State Certification and Licensure 

There has been much discussion in recent years regarding what qualities, skills, and 
knowledge an individual should posses in order to be an effective teacher. There has 
been even more debate about whether traditional teacher training programs and state 
certification and licensure systems result in effective teachers. Some policymakers argue 
that subject matter knowledge is the most important quality for teachers to possess, 
and therefore there should be few barriers for those with such knowledge to enter the 
teaching profession. Others have stated that pedagogy is just as important as subject 
matter knowledge because an individual has to be able to impart content and concepts 
to students. Both sides of the debate point to research that supports their position, but 
currently there is no consensus on which approach is most effective. 

The framers of the No Child Left Behind Act decided to use state certification and 
licensure systems as the cornerstone of the definition of “highly qualified.” Although 
some would argue that such systems are an imperfect measure of what constitutes an 
effective teacher, state certification is the only means currently available across all states 
to determine if teachers have met some qualifications before entering the classroom. 
This section discusses some of the problems with those systems and urges states and the 
federal government to take action to address them. However, an area that should be 
monitored over the next several years is whether or not the law’s emphasis on state cer- 
tification does result in better prepared teachers and increased student achievement. 

For teachers to be “highly qualified” they must, as a starting point, be fully certified. 
Many states already require prospective teachers to pass a test as part of their certifica- 
tion/licensure system, but as the American Federation of Teachers, the Education Trust, 
the National Commission on Teaching and Americas Future, and others have pointed 
out, the cutoff scores for passing these exams are often set so low that “there is no effec- 
tive standard for entry” into the teaching profession (National Commission on Teaching 
and America’s Future, 1996). Similarly, the American Federation of Teachers has observed 
that “in many fields, in many states, candidates who score at or above single-digit per- 
centiles qualify for a license, and very few states have cut-scores above the 25th percentile 
for any field.” Furthermore, the AFT found that these minimum requirements are often 
waived when states face teacher shortages (American Federation of Teachers, 2000). 

The Education Trust (2002) has raised concerned about the rigor of the exams. 
According to a recent report from that organization, four states (Alabama, Delaware, 
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Maine, and New York) use a test for licensing their teachers that 12 other states use to deter- 
mine entry into their teacher preparation programs. Eleven other states do not test teach- 
ers’ subject matter knowledge, and eight more have no statewide teacher assessment at all. 

Because oflow cut-scores, less than rigorous exams, or no exams at all, certification 
requirements in some states could be a very weak cornerstone for establishing “highly 
qualified” teachers. Similarly, states must use a “high objective state uniform standard of 
evaluation” to determine whether current teachers meet the federal criteria. This, too, 
could be a weak instrument for measuring quality if states do not set high standards for 
these evaluations. States and school districts also will be creating or adopting assessments 
for paraprofessionals. USED guidance has made it clear that these exams should evalu- 
ate paraprofessionals at a level equal to the second year of college. However, the Act 
does not give the U.S. Secretary of Education authority to review or approve the rigor 
or quality of a state s teacher certification system, exams, or other measures used to 
gauge the qualifications of teachers or paraprofessionals. Essentially, the Act demands 
higher quality but leaves the mechanism up to the states. 



Impact on Teacher and Staff Vacancies 

A final issue to watch is whether the new requirements make it harder for school dis- 
tricts to find and keep qualified teachers or instructional aides. States already face 
teacher recruitment and retention problems, and several state officials that we inter- 
viewed expect NCLB to make these problems worse. Setting more stringent qualifica- 
tions may discourage some prospective teachers from entering the field and could drive 
some veteran teachers out of teaching, especially secondary school teachers who teach 
several subjects and would have to meet requirements in each of those areas. States ex- 
pected this to be a problem primarily in rural areas but also in some inner city schools. 

Teachers who complete additional coursework to become highly qualified in a sec- 
ondary specialty may, understandably, expect higher salaries. School districts may have 
to compete for the best qualified candidates, increasing pressure to raise teacher salaries. 

The Elk Grove, California, case study at the end of this chapter shows how one district 
is addressing these challenges. 

Finding and keeping paraprofessionals could also become more difficult. State offi- ° 0 o 0 ° 
cials that we interviewed indicated that rural school districts may have difficulty find- 
ing paraprofessionals who meet the new requirements, as well as difficulty providing the 
necessary training for existing paraprofessionals. As one official said, “Its not as though 
there is a community college within an hour’s drive where the paraprofessionals could 
go to complete the necessary coursework.” An official from a mostly rural state report- 
ed that already this year, fewer school districts had proposed in theirTitle I applications 
to use Title I funds for paraprofessionals. This official saw this as a positive trend, in that 
more Title I instruction is being provided by teachers rather than aides. 

States and school districts have also expressed concerns about potential hidden costs 
of the paraprofessional requirements. Paraprofessional jobs tend to be low-paying. Once 
instructional aides receive an associate’s degree, many will expect to be paid more and 
to receive pensions or other benefits that school districts don’t typically provide to para- 
pro fessionals. This could put school districts in a position of having to raise wages or 
risk losing paraprofessionals to better-paying jobs. 
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Conclusion 

States are moving ahead to implement the teacher and paraprofessional qualifications 

requirements of No Child Left Behind, and are making good faith efforts to comply 

with the legislation, even though many state education agency personnel feel that the 

information provided by the U.S. Department of Education has been slow in coming 

or incomplete. States are in the process of revising current systems or building new 

ones to collect information on teachers’ qualifications and assignments. School districts 

have been informed of the new teacher requirements, and states will be monitoring 

their compliance during the school year. Similar activities are proceeding at the state 

96 and local level with regard to paraprofessionals, but states and school districts are not as 

o far along in their data collection efforts. 

oo A 

In years to come, policymakers should pay attention not only to how well states 
and school districts are complying with the teacher and paraprofessional requirements, 
but also to the quality and rigor of state certification/licensure systems for teachers. 
Attention should also be paid to whether or not these requirements are resulting in 
increased student learning. 

Boosting the qualifications of a large number of teachers and paraprofessionals is 
certain to be an expensive undertaking, and the federal government should share these 
costs. The Congress and the White House have an opportunity with the fiscal year 2003 
appropriations bill to send a signal to states and school districts that the federal gov- 
ernment will be a partner in this endeavor by providing significant funding for profes- 
sional development activities. 

The Congress and the White House should also consider ways to provide states 
incentives to increase the rigor of their certification systems. Federal funds under both 
Title II of the revised ESEA and Title II of the Higher Education Act can be used to 
improve state certification and licensure systems. But under Title II of the Higher 
Education Act, this activity is one of seven allowable activities, and under Title II of 
ESEA, it is one of 18 possible uses of funds. During the 108th Congress, the Congress 
and the White House will be considering revisions to the Higher Education Act. A pro- 
gram that provides incentive grants to states for the sole purpose of increasing the rigor 
of their teacher certification requirements should be considered. 
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CASE STUDY 

ELK GROVE UNIFIED SCHOOL DISTRICT, CALIFORNIA 



Elk Grove is well prepared to meet the NCLB teacher requirements because the 
district has strengthened its teaching force through such strategies as ongoing 
professional development and mentoring, "grow your own" teacher programs, 
and incentives for skilled paraprofessionals to become certified teachers. 

Although school districts across the nation, are facing a variety of issues as they 
begin to analyze the implications of the “highly qualified” provision of the No Child 
Left Behind Act, some have already been working to improve the quality of their 
instructional staff. In California, teachers must meet a rigorous standard in order to be 
awarded a teaching credential. The requirements include the following: a bachelor’s 
degree or higher, completion of an approved professional development program that 
includes successful student teaching, passage of the California Basic Skills Test 
(CBEST), passage of a subject matter competency test, and subject matter course work 
plus a course in the U. S. Constitution. California has reciprocal agreements with some, 
but not all states, because requirements are so different from state to state. 

In some California districts, the philosophy, focus for hiring, and past practices have 
alleviated much of the anxiety associated with the onset of the NCLB requirements. 

The Elk Grove Unified School District in Sacramento County views the issue of 
“highly qualified teachers and staff” as broader than just complying with the new fed- 
eral requirements, because the district has been creating and implementing support pro- 
grams for teachers and other staff for many years. The focus has gone beyond merely 
recruiting and hiring first-class applicants to encompass improvements in ongoing 
training, coaching, mentoring, and support services for teachers, administrators, and 
support staff, including incentives to retain excellent employees. A rapidly growing K- 
12 district of 52,000 students, Elk Grove is in the somewhat enviable position of already 
having a large number of “highly qualified” teachers. More than 300 teachers are hired 
each year; yet of the 2,996 teachers in the district in November 2002, only 35 had not 
met all aspects of the new federal and state qualifications. The minute percentage of O q q o 
teachers in this status are all licensed and meet most state requirements, but they are on 
emergency short-term permits because they do not meet all California requirements. 97 

Two years ago, there were 95 teachers in Elk Grove who were on emergency per- 
mits; last year there were only 71, and this year there are 35 — a reduction in three years 
from nearly 4% of all teachers to l%.The reasons why there are 35 are threefold. First, 

15 of the 35 are special education teachers, a critical shortage area. Some of these teach- 
ers are tenured, highly qualified professionals who have taught for many years but have 
not yet completed all requirements for this special field. Second, 12 of these teachers 
are secondary school teachers who may teach three or four periods in their field of cer- 
tification but must teach one or two periods out of their field because of scheduling 
and shortages. Third, the remaining 8 teachers are elementary teachers — including 
some who entered the teaching field under alternative certification programs — who 
have not yet completed the final phases of the requirements to be considered “highly 
qualified.” 
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With the exception of special education and some math/science teachers for which 
there is an extreme shortage, there will be far fewer than 35 teachers on emergency sta- 
tus next year. 

Elk Grove has made this progress even though it is a diverse district with a signifi- 
cant degree of poverty. Forty-four percent of the district s students are eligible for free 
and reduced-price lunches, and the concentrations of poverty in its Title 1 schools range 
from 60% poverty to over 90%. Elk Grove families come from 90 different countries, 
and the students list 77 languages as their primary or first language. The October 2002 
count of English language learners in grades K-12 was 11,764, or 22% of the dis- 
trictwide enrollment. The largest language groups include Spanish, Hmong, Vietnamese, 
98 Russian, Hindi, Filipino, Cantonese, and Punjab. Racial-ethnic balance is also diverse 

o with Asian, African American, and Hispanic students making up more than 50% of K- 

o o 12 enrollments. 

Why is Elk Grove in such a strong position as regards the qualifications of its teach- 
ing force? Teacher recruitment and training have been a priority for the past two 
decades because of rapid residential growth in the 320-square mile district, where hous- 
es continue to replace pastures, crop fields, and orchards. Close working relationships 
with colleges and universities, a very attractive salary schedule, excellent benefits and 
working conditions, ongoing and intensive professional development, a friendly “small 
town” atmosphere (despite the growth), and well-regarded schools have made Elk 
Grove a place sought out by families. Most elementary and middle schools are on four- 
track, year-round schedules because of the growth, so the new school year begins in 
July — which means that hiring takes place in the spring. 

Elk Grove has also made strong efforts to grow its own teachers. Hundreds of high 
school students have been exposed to the profession through an active Future Teachers 
club, and hundreds have come back to teach in Elk Grove schools. In addition, the dis- 
trict developed a Teacher Education Institute in partnership with San Francisco State 
University, which supplies as many as 100 teachers to the district each year. Applicants 
need a bachelors degree to enter the Institute. The program includes a full year of 
intensive course work, training, mentoring, and student teaching with highly qualified 
teachers. Graduates of this program continue their training through a Beginning 
Teacher Support and Assistance (BTSA) program, and many go on to mentor other 
new teachers. 

Para professionals are also a target of professional development in Elk Grove. In the 
district s Title I schools, there has been a steady movement to replace paraprofessionals 
with teachers. Many paraprofessionals with a demonstrated ability to work effectively 
with students have been encouraged and assisted over the years to complete college and 
obtain teaching certification. These teacher candidates receive extensive training in 
reading and math teaching methods through Elk Groves professional development 
department; Cosumnes River College, the local community college, provides course- 
work, especially for paraprofessionals in pre-kindergarten programs. The district will be 
continuing this partnership with the community college to create new programs for 
existing paraprofessionals, including those who provide bilingual services, to help them 
complete their two-year training programs and obtain an associate s degree. 

To address teacher shortages in hard-to-fill subjects like math, science, and foreign 
languages, Elk Grove plans to offer signing bonuses of $2,500, a strategy that has helped 
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in the past. The district also plans to offer extended contracts with higher pay. For 
example, a year-round middle school may need a teacher only for an additional period 
of math, but finding a qualified math teacher who is willing to work for a 20% con- 
tract is not easy. In the past, the district might have used an English teacher with a col- 
lege minor in math who was on contract for 80% of the time to fill that spot, but that 
individual would have needed an emergency credential and would not meet the “high- 
ly qualified” definition. Under the extended contract option, the district could pay an 
existing math teacher to teach six periods instead of five, with a contract of 120% 
instead of 100%. 

Other California districts have developed other approaches to build a highly qual- 
ified teaching force. Sacramento City Unified School District has undertaken a major 
recruitment effort in the Philippines, which brought 24 new teachers to the district this 
year; the district is helping them become fully credentialed.The Philippines was cho- 
sen for recruitment because its English-speaking educational system is similar to that of 
the U. S. and teacher candidates have already mastered English. 

Small rural districts sometimes face special problems due to their size. One small 
K-12 district in northern Wisconsin has experienced great difficulty finding and keep- 
ing special education teachers and reading specialists. The lack of competitive pay in 
small rural districts exacerbates the shortages typically found in these areas of special- 
ization. Similar situations have been reported in Alabama and Massachusetts. One strat- 
egy reported in a small district was to encourage retired qualified teachers to return to 
teach one or two periods a day in math or science. Overall, the greatest concern report- 
ed was that of finding enough qualified special education teachers. 
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Source: Elizabeth Pinkerton , Director of State and Federal Programs , Elk Grove 
Unified School District , Sacramento County, California. 
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CHAPTER 5 



Using Scientifically Based 
Research to 
Improve Education 

KEY FINDINGS 

0 Although numerous programs in the No Child Left Behind Act require schools to 
use scientifically based research (SBR) to inform their classroom practices, the 
U.S. Department of Education so far has provided guidance on implementing 
this provision for only certain programs, including Reading First, Comprehensive 
School Reform, and the Title II State Program to Improve Teacher Quality. State 
officials interviewed gave the Department low marks for its overall guidance, or 
lack thereof, on scientifically based research. 

0 States are not as far along in implementing the new requirements for scientifi- 
cally based research as they are with other major components of the new law. 

This seems to be mostly because several states were waiting on the federal gov- 
ernment to provide more definitive guidance before they moved ahead, and a 
few felt they lacked expertise on this issue. Some states, however, did not real- 
ly want more guidance from the Department because they feared it would be 
too restrictive. 

m Many Title I state directors report that the Reading First program in their states 

is taking the lead in describing scientifically based research. Since the Reading °0° 

First program uses a special definition of scientifically based research that is less ioi 

rigorous than the general definition that applies to other NCLB programs, it may 
not be prudent for Title I programs to rely on Reading First materials because 
they run the risk of not being in compliance with the SBR provisions that apply 
to Title I. 

m Some state officials, researchers and educators are concerned that strict adher- 
ence to the law's general definition of scientifically based research will produce 
few programs or approaches that meet it. 
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The Prominence of Scientifically Based 
Research Throughout the Act 



O ne of the major ways that the No Child Left Behind Act seeks to improve 
the quality of public education is by requiring educators to use programs 
and practices that are based on scientific research. The point of these re- 
quirements is to base school reform efforts less on intuition and more on research- 
based evidence. Although the testing and accountability provisions of the NCLB Act 
102 have attracted more attention, the numerous mentions of scientifically based research 

0 C> 0 throughout the law may also strongly influence what goes on in classrooms. 

O r u o 

The term “scientifically based research” appears more than 100 times in the No 
Child Left Behind Act, in a range of programs large and small. In Title I, Part A, the use 
of scientifically based research must be part of the plans developed to raise achievement 
in schools identified as needing improvement. Scientifically based research must form 
the basis for the technical assistance and support provided to low-performing schools 
by local school districts and by state-established support teams. And it must also be the 
basis for the reform strategies used in Title I schoolwide programs and the instruction- 
al strategies used to raise achievement in Title I targeted assistance schools. 

In the Reading First program authorized by NCLB, scientifically based research 
plays a prominent role in the statement of purposes and many other sections. This pro- 
gram will award $5 billion over the next six years to states and school districts for pro- 
grams to teach reading using scientifically based methods. And in the Comprehensive 
School Reform Program also authorized by NCLB, school districts seeking funds for 
whole-school reform must design programs that use instructional and school manage- 
ment strategies grounded in scientifically based research. 

Under the Title II teacher preparation and professional development program, state 
and local applications for funds must describe how the proposed activities will be based 
on a review of scientifically based research. Similar references to scientifically based 
research appear in several other programs of the Act, ranging from the Drug-free 
Schools program to the Language Instruction program for English language learners. 

The Reading First section of the Act has a separate definition of “scientifically 
based research” that applies only to that program. For all other programs under NCLB, 
the law contains a highly specific definition of scientifically based research, shown in 
Box 5-A. As discussed later in this chapter, this definition, if interpreted strictly, could 
have a significant impact on all the federal programs to which it applies. 



Why the Emphasis Now? 

The requirements for scientifically based research are the product of a growing desire 
among Members of Congress and Administration officials to ensure that federal funds 
are used for effective programs that have evidence of results. 
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BOX 5-A DEFINITION OF SCIENTIFICALLY BASED RESEARCH 
FROM THE NO CHILD LEFT BEHIND ACT 



The term "scientifically based research" — 

(A) Means research that involves the application of rigorous, systematic, and 

objective procedures to obtain reliable and valid knowledge relevant to 

education activities and programs; and 

(B) Includes research that — 

(i) Employs systematic, empirical methods that draw on observation or 
experiment; 

(ii) Involves rigorous data analyses that are adequate to test the stated 
hypotheses and justify the general conclusions drawn; 

(iii) Relies on measurements or observational methods that provide reli- 
able and valid data across evaluators and observers, across multiple 
measurements and observations, and across studies by the same or 
different investigators; 

(iv) Is evaluated using experimental or quasi-experimental designs in 
which individuals, entities, programs, or activities are assigned to dif- 
ferent conditions and with appropriate controls to evaluate the ef- 
fects of the condition of interest, with a preference for random-as- 
signment experiments, or other designs to the extent that those 
designs contain within-condition or across-condition controls; 

(v) Ensures that experimental studies are presented in sufficient detail 
and clarity to allow for replication, or, at a minimum, offer the oppor- 
tunity to build systematically on their findings; and 

(vi) Has been accepted by a peer-reviewed journal or approved by a 
panel of independent experts through a comparably rigorous, objec- 
tive, and scientific review. 



Source: Public Law 107-110 , the No Child Left Behind Act. 
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References to research-based approaches and methods in federal education pro- 
grams are not entirely new. Even when not explicitly stated, research was often assumed 
to be the basis for choices made by states, districts, and schools. For instance, the 
requirements for Title I schoolwide programs in the 1994 amendments to the 
Elementary and Secondary Education Act stipulated that instructional strategies should 
be based on “effective means of improving the achievement of children,” with the 
understanding that effectiveness could only be derived from a past record of accom- 
plishment (Improving America’s School Act of 1994, Section 11 14(b) (l)(B)(ii)). 

The point was stated more plainly in the original 1998 legislation (Public Law 1 OS- 
78) for the Comprehensive School Reform Demonstration (CSRD) program, which 
provided funds to support approaches based on “reliable research and effective prac- 
tices.” The U.S. Department of Education sought to help grantees implement this 
requirement by issuing a document entitled “Continuum of Evidence of Effectiveness,” 
which included rubrics to determine the relative strength of research data (USED, 

1998) . 

To provide additional guidance to states and school districts, the American 
Institutes for Research (AIR), under contract from several education groups, examined 
evidence of the effectiveness of 24 popular school reform models and found strong evi- 
dence of effectiveness for only 3 (deeming another 5 as promising) (Herman et al., 

1999) . However, according to the Comprehensive School Reform Database maintained 
by the Southwest Regional Development Laboratory (2002), many schools and districts 
have continued to select models with weak evidence of effectiveness. 

Thus, the Congress and the Administration remained concerned that federal funds 
were being used to support instructional strategies and approaches that lacked broad 
evidence for effectiveness and held little hope for improving student performance. 
Members of Congress opted for a much more specific definition of research and wove 
it throughout the No Child Left Behind Act. Advocates of this approach compare it to 
using clinical trials in medicine to reach impartial decisions about the best treatment. 

The report of the Committee on Education and the Workforce of the House of 
Representatives on NCLB cites the work of the National Institute of Child Health and 
Development (NICHD), which looked at the research basis of various reading programs. 
In testimony before the Committee, Reid Lyon of NICHD expressed concern that the 
term “research-based” meant different things to different people and called for adher- 
ence to scientific quality and criteria in the selection and implementation of reading 
programs for Title I children. Advocates for scientifically based research believe that 
much educational research lacks the rigor necessary to connect specific educational 
approaches and/or programs with specific educational outcomes for children. They con- 
tend that relying on scientifically based research will produce a better understanding of 
what works best for children and will help schools and districts make better decisions 
about curriculum and instructional approaches (House Report 107-63, pp. 288-89). 

Susan Neuman, the Assistant Secretary for Elementary and Secondary Education, 
elaborated on the Administrations rationale for the requirements in a speech on 
October 24, 2002. “I think the federal government in the past has done a little of this 
and a little of that,” Neuman said. “It seems like we were into a new trend every other 
year. NCLB is a bold change in the way we do business”(Balta, 2002). 





At the same time, the requirements for scientifically based research have generated 
controversy as educators have raised concerns that the provisions will be interpreted in 
a way that imposes a specific ideology on public schools, will yield just a small number 
of acceptable practices, will give the federal government too much control over cur- 
riculum and instruction, and other issues 



Within the U.S. Department of Education, different offices are responsible for imple- 
menting the various programs of the No Child Left Behind Act, so the Departments 
efforts to implement the scientifically based research provisions have varied from one 
program to another. The Department has issued guidance about implementing scien- 
tifically based research in the Reading First program, the Comprehensive School Re- 
form (CSR) program (the successor to the Comprehensive School Reform Demon- 
stration Program), and Improving Teacher Quality State Grants, but not for Title I 
beyond regulations that simply repeat the statutory requirements for SBR.The guid- 
ance issued for Reading First, CSR, and Improving Teacher Quality closely parallels 
the statute, with some elaboration on the types of questions used to evaluate the scien- 
tific basis of research. A comparison of the guidance issued by Reading First and CSR 
follows. 

In addition to program-specific efforts, the Department, in November, issued a 
report from the Coalition for Evidence-Based Policy (2002) that seeks to further sci- 
entifically based research. The report calls for more support for scientifically based 
research in education. It is the result of a joint effort between the Department and the 
Coalition. 



Although there are several references in the Act to scientifically based research in 
Title I, Part A that would appear to significantly affect the program’s operation, the 
Department has not yet issued any guidance or regulations on scientifically based 
research for this program, beyond the language in the statute. This includes the final reg- 
ulations for Title I released on November 26, 2002. 

Comprehensive School Reform Program 

The No Child Left Behind Act strengthened the requirements to use research-based 
evidence in local programs funded by the Comprehensive School Reform program. 
Under the law, school districts may use CSR funds only for proven methods and strate- 
gies for student learning, teaching, and school management that are grounded in scien- 
tifically based research and in effective practices that have been replicated in schools. In 
order to receive federal support, a program, method, or strategy must meet one of two 
standards: (1) it must have been found, through scientifically based research, to have 
significantly improved achievement of children, or (2) it must have strong evidence 
that it will result in significant improvement in achievement. 



U.S. Department of Education Actions 



Title I, Part A 
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In August 2002, the Department issued guidance for the CSR program. The guid- 
ance makes clear that the “strong evidence” standard, the second option mentioned in 
the law, is less rigorous than the “scientifically based research” standard, in that it lacks 
the broad research base included in the criteria for the latter term. The guidance states, 
however, that strong evidence is still research based, even though it does not meet all 
these criteria, and notes that the strong evidence standard should be used “in the 
absence of scientifically based research” (USED, 2002a). 

The August guidance summarizes the laws definition of scientifically based 
research and also suggests a series of questions that states, school districts, and schools 
can use to evaluate whether the research supporting the program, method, or strategy 
they want to fund meets the criteria for being scientifically based. Box 5-B gives a few 
c o° c> Q sample questions from the guidance, grouped according to the six elements of the gen- 

eral definition of scientifically based research included in the Act. 

Although this guidance was developed specifically for CSR, the questions suggest- 
ed under the topics are generic and could be used by anyone seeking to evaluate the 
scientific base of research. 



Reading First 

More than any other program in NCLB, Reading First is focused on the use of scien- 
tifically based research. However, the definition of scientifically based research that ap- 
plies to this program is different from, and less rigorous than, the definition that applies 
to all other programs in the NCLB Act. The Reading First definition has four, rather 
than six, main elements. Two elements mentioned in the more widely applied defini- 
tion — those related to the research design and sufficient detail and clarity to allow for 
replication — are not included in the special Reading First definition. In some ways, 
this more limited definition of scientifically based research is akin to the “strong evi- 
dence” standard included in CSR and discussed above. 

In March 2002, the Reading First office of the U.S. Department of Education 
issued guidance for its program. The guidance includes generic questions to test evi- 
dence for scientifically based research, as well as items specific to reading in the early 
grades. For those elements of the definition that are the same for both Reading First 
and CSR, the guidance issued by the two programs differs in content and emphasis. 

Like the guidance for the CSR program, the guidance for Reading First suggests 
questions, grouped around the elements of the statutory definition, that schools should 
consider using to test whether research evidence is scientifically based. In stating the top- 
ics, the Reading First guidance uses more verbatim language from the statute than was 
used by CSR. Box 5-C shows the topics and some sample questions from the guidance. 

The suggested questions in the Reading First guidance for these four common ele- 
ments differ substantially from those in the CSR document. For instance, under the first 
topic, Reading First suggests asking if the research was carefully designed to avoid 
biased findings and unwarranted claims of effectiveness. The CSR guidance does not 
mention this question, but it does include questions regarding the degree to which the 
study was conducted in a consistent, disciplined, and methodological manner; whether 
the data were obtained using observation or experiment; whether the research was 




BOX 5-B SAMPLE QUESTIONS FROM U.S. DEPARTMENT OF EDUCATION 
GUIDANCE FOR THE COMPREHENSIVE SCHOOL REFORM 
PROGRAM THAT CAN BE USED TO EVALUATE WHETHER 
RESEARCH IS SCIENTIFICALLY BASED 



1. Systematic and empirical methods 

Does the research have a solid theoretical foundation? Were the data 
obtained using observations or experiments? 

2. Rigorous data analyses 

Did the research test the stated hypotheses and justify the general 
conclusions drawn? Did the research minimize alternative explanations for ob- 
served effects? 

3. Reliable and valid data collection 

Did the data result from a study involving multiple investigators in a 
number of locations? 

4. Strong research design 

Does the experiment describe a random assignment experiment in which 
subjects are assigned to different conditions with appropriate controls? 

5. Detailed results that allow for replication 

Are the results of the research sufficiently detailed so that replication of the 
design is possible? 

6. Results that are subject to scrutiny 

Has the research been reviewed by unbiased experts who were not a part of 
the research study? 



Source: U. S. Department of Education, Guidance for the Comprehensive School 
Reform Program, August 2002. 





BOX 5-C SAMPLE QUESTIONS FROM U.S. DEPARTMENT OF EDUCATION 
GUIDANCE ON READING FIRST TO CONSIDER WHETHER 
RESEARCH IS SCIENTIFICALLY BASED 



1 . Use of rigorous, systematic and empirical methods. 

Does the research clearly delineate how it was conducted , by whom it was conduct- 
ed, and on whom it was conducted? 

2. Adequacy of the data analyses to test the stated hypotheses and justify the general 
conclusions. 

Does the research present convincing documentation that the observed results were 
the result of the intervention? 



3. Reliance on measurements or observational methods that provided valid data across 
evaluators and observers and across multiple measurements and observations. 

What procedures were in place to minimize researcher biases? 

4. Acceptance by a peer-reviewed journal or approval by a panel of independent ex- 
perts through a comparably rigorous, objective and scientific review. 

Has the research been carefully reviewed by unbiased individuals who were not part 
of the research study? 



Source: U.S. Department of Education, Guidance for the Reading First Program, March 2002. 
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grounded in data that are factual rather than opinion based (somewhat similar to the 
question about bias in the Reading First guidance); and whether the research findings 
are supported by tangible, measurable evidence. 

It appears from the guidance that these two programs do not entirely agree on its 
meaning. This is important because, in the absence of guidance from Title I, states have 
informed us that they will refer to the guidance issued by Reading First and, to a less- 
er degree, by CSR, in implementing Title I. States that rely on Reading First guidance 
run the risk of not being in compliance with Title I requirements regarding scientifi- 
cally based research. Further, it is in the Departments best interest to ensure that gener- 
ic (as opposed to program specific) guidance on the common elements of this topic is 
the same for all programs. 

There is also some evidence that the Department intends to interpret the scientifi- 
cally based research requirements of Reading First rigorously. The Department returned 
many of the Reading First applications submitted by the states, with instructions for 
states to make revisions. States were told that they would need clear evidence that local 
grant recipients will use the money only for scientifically based reading programs and 
that the recipients will have to show evidence that their approaches actually improve stu- 
dent achievement (Manzo, October 2, 2002). 



What Works Clearinghouse 

One of the ways in which Congress sought to help states, districts, and schools imple- 
ment the scientifically based research requirements was to authorize a clearinghouse 
that will set standards for high-quality research in education and then determine which 
studies meet those standards. This information is meant to help educators, parents, and 
others figure out which programs and strategies work and are worth adopting. In Au- 
gust 2002, the Department awarded a five-year, $18.5 million contract to operate this 
clearinghouse to the Campbell Collaboration of Philadelphia and the American Insti- 
tutes for Research in Washington, D.C.The clearinghouse will provide the following 
databases: 

■ A registry of educational interventions that identifies potentially replicable pro- 
grams to enhance student outcomes and synthesizes research related to their effec- °o 0 

tiveness. A linked database will provide information about the evaluation studies on 

which the reviews are based. 

■ A registry of evidence-based research reviews of educational approaches and poli- 
cies. 

H A registry of scientifically rigorous reviews of test instruments for assessing educa- 
tional effectiveness. 

H A registry that identifies evaluators and organizations willing and able to conduct 
evaluations of educational interventions. 

The clearinghouse has a technical advisory board and, during fall 2002, was solic- 
iting nominations for topic areas to be included in the databases, such as third grade 
reading and cross-age grouping. The clearinghouse is also developing a set of standards 




119 



Center on Education Policy 



From the Capita! to the Classroom 



for review of items nominated for inclusion in the databases. Draft standards have been 
developed and public comment on them is being solicited. The Clearinghouse plans to 
have the standards in place by early 2003. The website for the clearinghouse is 
http://w-w-c.org. 
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State Views of Federal Guidance 

When we interviewed state officials, we asked them to rate the guidance available from 
the U.S. Department of Education on the main issues treated in this report. The guid- 
ance for scientifically based research was the area that respondents rated the lowest — 
probably because of what they saw as a lack of guidance about this topic. A sizeable 
share of the 37 respondents (43%) felt the quality of guidance on scientifically based 
research was “poor,” the lowest rating, and more than half of the 39 respondents (54%) 
rated the timeliness of this guidance as poor.The total responses are shown in Figures 
5-A and 5-B. 



State Activities 

Most state officials we interviewed generally felt that the emphasis on scientifically 
based research in the No Child Left Behind Act was a positive change. States said that 
bringing research and proven methods to bear on educational problems was promising 
because schools and districts would be forced to think more deeply about why they 
were choosing to implement specific programs in their schools. At the same time, states 
expressed some concerns about the provisions, described later in this section. 

Except for preparing applications for the Reading First program, most states have 
done little specific planning to apply the scientifically based research requirements to 
Title I or other major grant programs. This lag at the state level seems at least partly 
attributable to the absence of guidance from the U.S. Department of Education about 
how the requirements should be applied to Title I and most other federal programs. 
While some states are awaiting more guidance from the Department, other states, as 
noted below, do not really want more guidance because of what they fear it could entail. 



State Implementation Actions 

Our interviews revealed that most state Title I directors were aware of the requirements 
for scientifically based research and its potential importance, but they had done little to 
implement them. So far, state Title I offices are much more focused on other provisions 
of NCLB that demand immediate action, such as assessment and accountability, and are 
paying little attention to the requirement for scientifically based research. 

Some state Title I officials told us that in the absence ofTitle I-specific guidance, 
they have sought out information on scientifically based research from the Reading 
First Program and the Comprehensive School Reform program, In particular, many 
states are looking to their Reading First program to take the lead in implementing the 
requirements for scientifically based research, often in collaboration with other state- 
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FIGURE 5-A STATE RATINGS OF QUALITY OF WRITTEN FEDERAL 
GUIDANCE ON SCIENTIFICALLY BASED RESEARCH 




Source: Center on Education Policy state interviews 



FIGURE 5-B STATE RATINGS OF TIMELINESS OF WRITTEN FEDERAL 
GUIDANCE ON SCIENTIFICALLY BASED RESEARCH 




Source: Center on Education Policy state interviews 
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level programs. At the time of our interviews, however, only 11 states had received 
approval of their Reading First applications (not all states had yet applied). The other 
states were reluctant to discuss how they planned to implement the Reading First pro- 
gram, including scientifically based research, until their applications had been approved 
by the federal Department. 

More importantly, ifTitle I relies heavily on Reading First, this could be problem- 
atic because of the differences already noted between the general definition and the 
Reading First definition of scientifically based research. It is unclear whether states are 
aware of this difference and how they will proceed to address the two items of the gen- 
eral definition that are not in the Reading First definition. For instance, one state said 
that the Reading First office in the state was compiling a list of programs that would 
(P 0 o meet the definition of scientifically based research and that this fist would be used by 

Title I as well as other reading programs in the state. In another state, Title I officials 
said they were reluctant to send out Reading First materials because districts were 
responding angrily, asking if they had to use those materials. 

State Title I directors seem to be responding to the scientifically based research 
requirements in three different ways: 

□ Many states had been awaiting further guidance from the Department before 
actively planning to implement the requirement. These states are concerned that 
any actions they take could be at odds with the interpretations eventually made by 
the Department, so they are doing little except to distribute the statutory provision 
to their school districts. One state official, for instance, said that it was unclear what 
scientifically based research meant in a practical sense and that the state would like 
some help on this. Another state official made a suggestion (an atypical view among 
the states) that the Department should develop common criteria to validate which 
programs are based on scientifically based research and should identify some exem- 
plars. States could then apply the common criteria to local programs. 

M Several states are pleased with the lack of guidance in Title I and would be happy 
if it stayed that way.They fear that any guidance will only reduce the flexibility they 
believe they now have to interpret the statute at the state level. It does seem that at 
least some states would interpret and apply the provisions less rigorously than the 
Department would. One state official, for instance, commented that the lack of 
guidance was not “a bad thing” and expressed the hope that the Department would 
“leave it alone ” Another state official commented that scientifically based research 
“is not a priority for us” Yet another state official expressed anxiety that the 
Department is “just waiting to zing us on this”; rather than providing guidance to 
local districts, the state is referring them to the USED website for assistance on the 
topic. The same state person said that if federal guidance remains vague, the state 
would not strictly enforce the provision. 

□ A few state departments said they lack the expertise and/ or time necessary to help 
districts with this issue. They are hoping that information from the What Works 
Clearinghouse will be helpful. 
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While some state personnel said they planned to rely on the What Works 
Clearinghouse for information, others indicated that they expected help from the fed- 
erally funded regional education laboratories and comprehensive centers. A few states 
said that they planned to use an approved list of programs or methods, and a few 
planned to issue criteria for districts to use in determining at the local level if approach- 
es were rooted in scientifically based research. However, no state had done this yet at 
the time of our interviews. 



State and Local Concerns 

State officials and local educators have expressed some concerns about how the federal 
government will interpret and enforce the requirements for scientifically based re- 
search. Some of these concerns deal with the political and ideological implications of 
the requirements, while others deal with practical and technical implications. 



1 . Some educators are concerned that the requirements for scientifically 
based research could be interpreted to severely limit classroom teaching 
methods and materials. 

The main concern about scientifically based research raised by the states we inter- 
viewed was that the NCLB requirements could lead the Department to issue a limited 
list of approved programs or instructional methods that states and school districts would 
have to use. For example, the programs or strategies that pass muster with the What 
Works Clearinghouse could come to be perceived as the “approved” list. 

Some state officials and local educators are apprehensive that the Bush 
Administration intends to use the new authority to impose a specific ideology on the 
public schools, especially in reading. One state, for instance, noted in our interviews that 
scientifically based research can be interpreted to include the research the 
Administration likes — and exclude the research it opposes, by labeling it as unscientif- 
ic. Particular controversy has arisen about whether the Administration intends to 
impose strict phonics-based teaching methods on the schools by approving or denying 
federal funding through the Reading First program and perhaps others. Phonemic 
awareness and phonics are two of the five essential elements in NCLB that must be part 
of every local reading program (the others are vocabulary development, reading fluen- 
cy, and reading comprehension strategies). Assistant Secretary Neuman has denied the 
charge that the Department will impose strictly phonics-based approaches, saying, “We 
can’t do that nor would we want to” (Strauss, 2002). 

Some states were further concerned that a strict interpretation of the law could end 
up narrowing their choices to a handful of off-the-shelf programs that have reputations 
for having a solid research base. After attending workshops on the reading legislation held 
in February 2002 by the U.S. Department of Education, some states said they were get- 
ting a subtle message that they could improve their chances of receiving Reading First 
funds if they use certain commercial products and materials (Manzo, February 20, 2002). 

Some states have questioned whether the scientifically based research requirements 
represent excessive federal involvement in state and local decisions about curriculum 
and instruction. 
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States, especially those with histories of local control, are concerned about where 
the new requirements could lead. One state, in which school districts have a great deal 
of local autonomy, thought that it could create problems if the state suggested programs 
for the districts to use that had a scientific research basis. Another state gave examples 
of reading programs to schools and districts, which upset district administrators and 
school principals, who asked: “Do we have to have this program?” 
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2. States are concerned that the definition of scientifically based research is 

so precise that only a few education programs will meet it. 

Many states we talked to feared that a strict application of the laws definition of scien- 
tifically based research will make it hard to find programs and approaches that fully 
meet it. For example, the definition shows a preference for “randomized” studies, in 
which large populations of students are randomly assigned to two groups: one that re- 
ceives the program or teaching method being tested, and another that receives whatever 
programs or methods their schools normally use. These kinds of studies are expensive 
and not very common in education because they are complex to set up in a school situ- 
ation and because it can be difficult to get families to agree to their children’s participa- 
tion. It is also difficult to maintain control groups, because children may be receiving 
educational services from another source not being studied and pupil mobility affects 
both treatment and control groups. In addition, it is hard to justify denying potentially 
helpful services to children. Some researchers also contend that even the most rigorous- 
ly designed random studies cannot really identify why a particular practice does or does 
not raise achievement and that so many variables go into learning, one can’t assume that 
something that works in one classroom will work in another (Traub, 2002). 

Recognizing that few programs may meet the definition, the Coalition for 
Evidence-Based Policy, a foundation-funded organization of well-known researchers, 
has called on the Department to launch a major effort to build the knowledge base on 
the effectiveness of educational interventions through randomized controlled trials 
(Coalition for Evidence-Based Policy, 2002). 

Educators and researchers have also noted that the definition in the law assumes 
that there is a clear “yes” or “no” answer to the question of whether a particular prac- 
tice is scientifically based, when actually the answer is often a matter of degree. Some 
practices have more supporting evidence and come closer to the ideal, while others 
have little evidence and are farther from the ideal. Perhaps the question should be, “If 
the definition sets the ideal, how close to that ideal does any specific program or 
approach come?” 



Issues to Watch 

Based on the actions federal and state governments have taken and the concerns states 
have raised, the Center on Education Policy has identified several issues that should be 
watched in the coming years to see how they play out at the state and local levels. Some 
of these pertain to federal activities and others to state activities. We have posed them 
as questions to be asked as implementation progresses: 
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m Will the Department provide specific guidance on how the requirement for scien- 
tifically based research applies to the Title I program? 

^ How will the registries being compiled by the What Works Clearinghouse, espe- 
cially the one dealing with educational interventions, be used? The emphasis the 
Department puts on the results of the Clearinghouses review could signal how 
strictly the Department intends to enforce the provision. 

m In the absence of guidance for Title I, will the Department cite states and school 
districts for being out of compliance if those states and districts follow Reading 
First SBR guidance? 

a Are there sufficient programs and approaches that meet the definition for states and 
local districts to use? If only a few approaches are found acceptable, the issue of fed- 
eral intrusion into instructional areas traditionally left to states and local districts 
becomes critical. 

O In states with strong traditions of local control, will state departments of education 
be able to influence local districts to implement the provision in a way that’s 
acceptable to the U.S. Department of Education? As noted earlier, several states 
have already encountered problems in this area. 

E9 Will scientifically based research have a significant impact on the programs and 
approaches used in Federal programs, especially Title I? Clearly, that is its intention, 
but it is difficult to gauge how this issue will unfold, because the Department and 
the states have not taken definitive actions yet. 

After analyzing the information from our interviews and other research, we urge 
the federal government not to use the scientifically based research requirements to 
impose a set of specific practices or methods on teachers or to overstep the prohibitions 
in NCLB against federal intrusion in curriculum and instruction. At the same time, the 
field of education needs to pay more attention to research— based evidence about 
whether a particular practice improves student learning and could benefit from more 
clarity about which programs and practices actually do what they purport to do. 

Educators can no longer treat all “evidence” equally and ignore solid evidence of effec- 
tiveness while using information based on less rigorous research to make decisions 
about curriculum and instructional practice. 
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Protocol for Review 
of State Applications 

STATE: REVIEWER: DATE: 



1) Is the state in compliance with standards, assessment, and accountability provisions 
enacted in 1994? □ Yes □ No 

If no, is the state under a waiver of time or a compliance agreement ? 

Date of expiration of waiver or compliance agreement 



2) Does the state plan meet the requirements of NCLB (has it been approved by the 
Department of Education)? □ Yes □ No 

If no, list the major areas in which the Department is seeking amendments or addi- 
tional information. 



3) For each of the first four goals required in the plan, list two or three major indica- 
tors of progress on which the state will report. 



4) Does the state claim to meet the requirement that it must have challenging content 
standards in reading/language arts for each grade 3-8? 

In mathematics for each grade 3-8? 

If not, what is the state's schedule for adopting them? 

117 

5) Does the state plan claim existence of standards in science that meets NCLB ? 

If no, does the plan include a timetable for development of the standards? 

What is the expected date of completion of the standards in science? 
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6) Does the state claim to already meet the requirement for assessments in 
reading/language arts in each grade 3-8 and for high school? 

□ Yes □ No 

In mathematics? 

If no, is a timetable for development of those assessments included? 

What is the date for completion? 
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7) What is the additional indicator the state plans to use to measure progress in 
elementary and middle schools? 



8) Does the plan describe the statewide system of support the state will provide, 
including the use of school support teams? □ Yes □ No 



What are the two or three main examples of state actions included in the system of 
support? Include only specific items, not terms such as providing technical assis- 
tance. If there are no specific items included, say none. 



9) List the four or five major activities the state says it will conduct to ensure that al 
teachers in the state are high qualified. Activities should be specific, not a general 
statement of support for local school districts. 



10) List the three or four major activities that the state will undertake to ensure that all 
paraprofessionals (excluding those working with parents or as translators) meet 
the qualifications contained in NCLB by the 2005-06 school year. 



1 1) In the allocation of school improvement funds (section 1002(a)), does the plan de- 
scribe a formula for such allocation? □ Yes □ No 

If yes, what is the major factor in the formula (e.g. enrollment, extent of economic 
deprivation, etc)? 

If no, what are the major considerations in making allocations? 



12) Has the state estimated the cost of development of assessments to meet the re- 
quirements of NCLB? If so, how does this estimate compare with the allocation of 
federal dollars it will receive? 



13) Summarize the state plan in regard to supplemental services. How will it inform lo- 
cal education agencies? Does it include a list of providers or how such providers will 
be selected? Does it include a procedure to determine the amount of funds that 
will be available for each child that will be uniform throughout the state? 



14) Summarize the plan's discussion of public school choice in regard to Title I schools 
in need of improvement. 





ENGLISH LANGUAGE LEARNERS 

I. Questions for Reading First Application review 

1) What is the amount of the award? 

2) What are the four or five major activities or priorities for awards the state 
describes in its application? 

3) How does the state describe the manner in which Reading First and Title I ac- 
tivities will be coordinated? (This can be at the state, district, and school levels. 
For instance, at the state level, the state may review applications for Reading 
First in light of district Title I plans. Districts may require schools to show how 
activities will be coordinated or state that if a school receives Reading First 
funds, it will use Title I funds for mathematics and reading in upper elementary 
grades.) 
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Interview Guide 

*Not all states chose to respond to a II questions. In addition , reviewers sometimes var- 
ied the format and content of the interview depending on the state's responses. 



STATE: INTERVIEWEE: 

DATE: INTERVIEWER: 



I. ASSESSMENTS 

1 . Has your state provided districts with results on the state assessment for school 
year 2001-02? 

If yes, when was the data provided? 

If no, when will it be provided? 



2. (a) Using 2001-02 data, has your state identified districts and schools in need of 

improvement and in need of corrective action? 

If yes, what measure of adequate yearly progress did you use? 



(b) How many schools are in each category? 

(c) How does the number identified under the new Act compare with those 
identified the previous year? 

If no, when do you plan to identify the schools? 

(d) Do you anticipate that the new provisions regarding accountability and iden- 
tification of schools will increase or decrease the number identified the previous 
year? Can you estimate the extent of expected change? 

(e) What additional indicator does the state plan to use to measure the 
progress in elementary and middle schools? 
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3. Are you experiencing or do you anticipate any difficulties with disaggregating 
data as required under NCLB? If yes, what are the particular problems you 
face? 
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4. (a) Has your state developed a list of approved supplemental education service 
providers? If yes, when was it distributed to school districts? If not, when will 
such a list be available to school districts? 

(b) What are the key elements used or that will be used to gain approval? Ob- 
tain a copy of the list of providers. 

5. Did the state develop guidance for NCLB for LEAs? Did the state develop 
guidance for NCLB for schools? If yes, obtain a copy of guidance materials. 

6. (a) When did the state complete review and final approval of local applications? 
(b) When were awards made to local school districts? 



7. (a) What information and/or guidance did the state provide to school districts 

regarding public school choice under Title I? 

(b) Does the state have any information on the use of this option for the 2002- 
OS school year? If yes, obtain a copy of materials. 



8. (a) Has the state determined the cost of developing and implementing the aca- 

demic content and academic achievement standards and the assessments re- 
quired by NCLB? 

(b) Compare with allocation of funds for assessments. 



9. (a) How does the state define proficiency in reading/language arts and mathe- 

matics? 

(b) Has the definition changed as a result of NCLB? If yes, how? Has the defini- 
tion been strengthened or weakened? 



10. Does the state plan to make changes in its assessment system as a result of 
NCLB? If yes, what changes will be/were made? 



1 1 . How does the state plan to meet the goal of having all students performing at 
the proficient level by 2014? What are the three to five main things it will do? 
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12. (a) How would you rate written guidance provided by the U.S. Department of 
Education on assessment in regard to its quality? 

A. Excellent B. Good C. Fair D. Poor 

(b) How would you rate the timeliness of written guidance provided on 
assessment? 

A. Excellent B. Good C. Fair D. Poor 

13. (a) How would you rate other forms of assistance such as oral communication, 
the website, regional meetings in regard to its quality? 

A. Excellent B. Good C. Fair D. Poor 

(b) How would you rate the timeliness of other forms of assistance provided? 
A. Excellent B. Good C. Fair D. Poor 



tl. IMPROVE EDUCATOR QUALITY 

1 . Does the state have information on the number and location of teachers meet- 
ing the definition of highly qualified teachers in NCLB? If yes, please provide 
the information. If not, how is the state going about collecting the information? 

2. (a) Has the state changed its teacher certification policies in the past two years? 

If not, does it have any current plans to do so? 

(b) In your opinion, have the changes strengthened or weakened the certifica- 
tion? 

3. Does the state have a test of content knowledge that it uses to determine the o q 0 0 

adequacy of knowledge of content of prospective teachers? If not, is such a ° 

test under development at the state and/or local levels? Or is the state in the ^3 

process of selecting a commercially developed test? If so, which one, if known? 

4. Has the state established a reporting system for local districts to determine 
compliance with educator quality requirements? If a form is used, obtain a 
copy. 

5. How is the state ensuring that all teachers hired to work in programs supported 
with Title I funds after the first day of the 2002-03 school year are highly quali- 
fied? 



6. Does the state have information on the qualifications of paraprofessionals cur- 
rently working in Title I schools? If yes, how do the qualifications compare with 
the requirements of NCLB? If not, how does the state plan to gather this infor- 
mation? 
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7. How is the state ensuring that paraprofessionals hired after the passage of 
NCLB meet the qualification requirements of the Act? 



8. Has the state issued guidance on the educator quality provisions? If yes, obtain 
a copy. 



9. (a) How would you rate the written guidance issued by the U.S. Department of 

Education on this issue in regard to quality? 

A. Excellent B. Good C. Fair D. Poor 
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O O (b) How would you rate the timeliness of written guidance issued by the U. S. 

Department of Education? 

A. Excellent B. Good C. Fair D. Poor 



III. SCIENTIFICALLY BASED RESEARCH 

1 . Has the state issued any guidance on Scientifically Based Research (SBR)? If 
yes, obtain a copy of guidance materials. If not, does the state plan to issue 
guidance on SBR and if so, when? 



2. How does the state plan to ensure that this provision is met? For example, de- 
veloping lists of programs that meet the standard, use of the clearinghouse, re- 
view of local programs against some criteria, etc. 



3. (a) How would you rate federal guidance issued on SBR in regard to quality? 

A. Excellent B. Good C. Fair D. Poor 



(b) How would you rate the timeliness of guidance provided on SBR? 
A. Excellent B. Good C. Fair D. Poor 



IV. EARLY LITERACY 

1 . What are the priorities and criteria that the state has established to award 
grants under Reading First? 



2. If grants have been awarded, what are the three to five major activities local 
districts will carry out with Reading First funds? 



3. What are the two or three major ways states will ensure that Reading First 
funds are coordinated with those available to districts under Title I? 
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ENGLISH LANGUAGE LEARNERS 



1 . In the 2002-03 school year, does the state plan to test the English proficiency 
level of English language learners? 

If yes, what test will be used? 

If no, what are the plans to develop and administer an English language profi- 
ciency assessment? 



VI. GENERAL QUESTIONS 

1 . Do you believe that the NCLB Act will lead to improvement in the performance 
of all children and, in particular, children in high poverty, low performing 
schools? If so, give at least three major reasons why you hold this belief. If not, 
give at least three reasons why you think it will not result in improvement. 

2. What are the two to four major problems of implementation that your state 
faces in implementing NCLB? How do you plan to overcome each problem? 

3. (a) Among the changes made to ESEA by NCLB, what are the two to four that 
you find most favorable? Give reasons for each. 

(b) The two to four you find most troubling? Give reasons for each. 



4. (a) What are the two to three areas of NCLB which best complement state poli- 

cies? 

(b) What are the two to three areas of NCLB that greatly differ from state policies? 



CL 

°o° 



125 




134 



Center on Education Policy 



APPENDIX 3 



A New Federal Role 
in Education 



In January 2002, President Bush signed into law the “No Child Left Behind Act.”The 
central feature of this law requires the states to adopt a specific approach to testing and 
accountability, intended to lead to higher achievement for all children. The legislation 
sends the message that the federal government will be assuming a more forceful role in 
elementary and secondary education, one that makes unprecedented demands on states 
and local school districts-to raise academic achievement and to take direct action to im- 
prove poorly performing schools. The new law also requires states to raise the qualifica- 
tions for new teachers and verify the qualifications of current teachers. In exchange for 
meeting the new demands, poorer school districts will receive additional federal fund- 
ing, and all states and school districts will have greater flexibility in how they use federal 
funds. This summary covers the main provisions of the new statute. The legislation makes 
numerous other changes in federal K-12 education programs — too many to list here. For 
a more detailed description of the law and of the provisions described below, please 
check other sources such as the U.S. Department of Educations No Child Left Behind 
web site (www.nochildleftbehind.gov). 



Regular Testing in Key Subjects 

Currently, states must administer annual exams in reading/language arts and mathemat- 
ics at least once during grades 3 to 5; grades 6 to 9; and grades 10 to 12. The new law 
requires states to test students in more grades, using assessments developed or chosen by 
each state. These test results will be used to hold educators, schools, and districts ac- 
countable for student achievement. Since these testing requirements will entail addi- 
tional costs, they are contingent on the federal government providing a set amount of 
funding each year to help states cover the costs. 

■ By school year 2005-06, states must administer annual statewide tests in mathe- 
matics and reading/language arts to children in grades 3 through 8, and at least 
once during grades 10 tol2, and must provide individual student test scores. 

m By school year 2007-08, students must be tested in science at certain grade spans. 

B Starting in school year 2002-03, states must annually assess the English proficiency 
of students who are learning the English language. 

B Every other year, states must administer the mathematics and reading exams of the 
National Assessment of Educational Progress (NAEP) to a sample of their students in 
grades 4 and 8, with the federal government paying for the costs. NAEP is a nation- 
al testing program that tracks student achievement in core subjects. This requirement 
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is meant to serve as an independent check on the states’ own tests. Since the same 
NAEP exams will be given in every state, a comparison of NAEP results with a spe- 
cific state’s test scores could help determine the difficulty of a state test. 
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Test Design and Use 

The new law contains specific requirements about the features and uses of state tests. 

□ State tests must be aligned with the state’s academic standards and must produce 
results that are comparable from year to year. 

□ State tests must yield results that can be used to determine whether students are 
meeting the state standards and to help teachers diagnose and address students’ spe- 
cific academic needs. 



□ States must promptly provide test scores to local school districts by no later than 
the beginning of the school year after the test is given. 

When the law was signed in January 2002, only 9 states were at all close to fulfilling 
those requirements, according to surveys by the newspaper Education Week. 



Disaggregated Test Scores 

The new law requires every school, school district, and state to “disaggregate,” or break 
out, the average test results for certain groups of students, including: 

□ major racial and ethnic groups; 

□1 major income groups; 

□ students with disabilities; and 

□ students with limited English proficiency. 

This requirement is meant to highlight the relative achievement levels of these groups 
of students and to hold schools accountable for closing the achievement gap between 
African-American and Hispanic students on one hand, and Caucasian and Asian stu- 
dents on the other. 



Closing the Achievement Gap 

Using disaggregated test information, states are required to follow a precise timeline to 
close achievement gaps between different racial, ethnic, and income groups, and other 
groups noted above. Beginning after school year 2001-02, states have 12 years to move 
all groups of students to the benchmark set by the state for proficiency in mathematics 
and reading. States must set regular targets for increasing achievement over that period, 
using as a starting level the average achievement of the lowest performing group of stu- 
dents or schools in the state. 





Failing Schools 

Each school must test at least 95% of its students, and each group of students in a school 
must meet or exceed the annual objectives set for them. Schools receivingTitle I aid 
that do not reach state performance objectives will be subject to various forms of assis- 
tance, intervention, and other actions, depending on how long the failure persists. (Title 
1, the largest federal education program, provides aid to low-income schools to improve 
education for low-achieving children.) 

m If a school fails to meet performance objectives for two consecutive years, then in 
the third year, it must receive technical assistance from the district to help it 
improve, and its students will have the option to transfer to another public school 
in the district. 

El After the third consecutive year of failure, technical assistance to the school and 
public school choice will continue. In addition, students will have the option 
of using their share ofTitle I funds to pay for tutoring and other supplemental edu- 
cational services either from their own school or from a state-approved outside 
group, such as a for-profit company or a private non-profit entity. 

m After the fourth consecutive year, technical assistance, public school choice, and 
supplemental services will continue, but the failing school must also change its 
staffing or make another fundamental change. 

m After the fifth consecutive year, the governance of the failing school must be 
changed — for example, by converting it to a charter school, turning it over to a pri- 
vate management company, or having the state take it over. 



Report Cards and Parents' Right to Know 

Each school district must issue a report card to parents and the public that includes the 
following information: 

H The local report card must describe the state test results for students in the district, 
and compare the local results with those for the whole state. 

■ It must include test results for each school in the district, and compare each schools 

results with those of the whole district and the whole state. 129 

m It must list the schools in the district that are in school improvement (the term used 
to describe schools that are not raising achievement for all groups of students and 
that must follow the schedule set out above). 

States must issue similar report cards. 

Parents also have the newly-granted right to request information on the qualifications 
of teachers in a school, such as whether teachers are state-certified and licensed or 
whether they are teaching with provisional certificates. 



Teacher Qualifications 

By 2005-6, states must have highly qualified teachers in all their public school class- 
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rooms where core academic subjects are taught. States must take certain steps in the in- 
terim years to meet this goal. “Highly qualified” means that a teacher must be fully cer- 
tified or licensed, have a bachelors degree, and show competence in subject knowledge 
and teaching skills (generally demonstrated by passing a rigorous state test). The require- 
ments differ somewhat for new and already-hired teachers, and for elementary, mid- 
dle, and high school teachers. Also, after the beginning of the school year 2002-03, all 
new teachers hired whose salaries are supported by Title I program funds, must be high- 
ly qualified as must new teachers in schoolwide programs. 



130 Paraprofessional Qualifications 

o O°o o By January 8, 2006, all paraprofessionals supported by Title I funds who perform in- 
structional duties (including all such paraprofessionals in Title I schoolwide programs) 
must complete at least two years of higher education or meet a rigorous standard of 
quality, as determined by a test. This requirement took effect immediately forTitle I 
supported paraprofessionals hired after January 8, 2002. 



Flexibility 

In exchange for meeting these federal demands, the new law gives educators more flex- 
ibility in the use of federal money The main flexibility provisions include the following: 

■ School districts can shift up to 50% of the grants they receive under four federal 
categorical programs into any of the other three programs or into the Title I pro- 
gram. These four programs are for teaching improvement, innovation, technolo- 
gy, and safe and drug-free schools. 

■ In the Title I program, schools with a poverty rate of 40% may now use their Title 
I grants to improve education for all the children in the school, rather than just for 
the lowest-achieving students. Previously, only schools with a poverty rate of at least 
50% could operate these “schoolwide” projects. 

■ The new law authorizes several experimental programs to test what happens when 
more federal requirements are relaxed in a select number of states and school dis- 
tricts. 



More Funds for the Poorest Districts 

The new statute makes several changes in the method for distributing Title I funds, in 
order to direct additional funding to the poorest school districts. Even more significant- 
ly, the actual dollars to carry out this increase for low-income areas were appropriated 
by the fiscal year 2002 appropriations bill. It was important that both the authorizing 
law (the No Child Left Behind Act) and the appropriations bill had the same goals, be- 
cause in past years, funds have not always been appropriated for the more highly- target- 
ed sections of the Title I funding formula. 
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Other Programs 

The new legislation revises and extends many other federal aid programs for elementary 
and secondary education, including the Bilingual Education program, Impact Aid, the 
Safe and Drug-Free Schools and Communities Act, the 21st Century Community 
Learning Centers, and the education technology program. Two especially noteworthy 
new programs are the following: 

m The Teacher Quality initiative makes funds available to school districts to recruit 
and retain teachers and principals and provide them with professional development. 
This new program combines two formerly separate programs: the Eisenhower 
Professional Development program and the class-size reduction initiative. 

m The Reading First program helps school districts carry out comprehensive pro- 
grams to improve reading instruction. Grant recipients must conduct diagnostic 
testing of children in K-3 to determine who is at risk of reading failure. 



Immediate Action In School Year 2002-03 

To underscore the urgency of the new federal demands for accountability, the law in- 
cludes some important changes that will be effective in the school year 2002-03. 

a In fall 2002, as already noted, new teachers hired with Title 1 funds or teaching in 
schoolwide programs must be highly qualified. 

□ All Title I supported paraprofessionals who perform instructional duties and who 
were hired after January 8, 2002 must have completed at least two years of college 
or must meet a rigorous standard of quality as determined by a test. 

m In fall 2002, students in schools that have failed for a second year to meet the 
improvement provisions of the prior law will have the option of leaving the fail- 
ing school and enrolling in a different public school in the district. According to 
the U.S. Department of Education, students in an estimated 8,652 schools will 
qualify for this option.The local school board must pay for some or all of these stu- 
dents’ transportation expenses. 

m In fall 2002, students in an estimated 3,000 schools will be offered both the option 

of public school choice and of taking away from the public school system their per- 131 

pupil share of Title I funding (between $300 and $1,000 per child) and transferring 
that amount to a private company, religious institution, or non-profit organization 
to pay for after-school tutoring or other supplemental services. This provision 
applies to students in schools that have already been labeled as failing for three years 
under the previous federal law. 

13 In 2002-03, students who are learning the English language must be assessed to 
determine their English proficiency. 
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